[GitHub] [spark] AmplabJenkins commented on issue #27786: [SPARK-31034][CORE] ShuffleBlockFetcherIterator should always create request for last block group
AmplabJenkins commented on issue #27786: [SPARK-31034][CORE] ShuffleBlockFetcherIterator should always create request for last block group URL: https://github.com/apache/spark/pull/27786#issuecomment-594791573 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119319/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27786: [SPARK-31034][CORE] ShuffleBlockFetcherIterator should always create request for last block group
SparkQA commented on issue #27786: [SPARK-31034][CORE] ShuffleBlockFetcherIterator should always create request for last block group URL: https://github.com/apache/spark/pull/27786#issuecomment-594789989 **[Test build #119319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119319/testReport)** for PR 27786 at commit [`dc94365`](https://github.com/apache/spark/commit/dc9436573976e9ba227836dc7e1ce271cd91c7fb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594788986 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594789008 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119326/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
SparkQA removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594670456 **[Test build #119326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119326/testReport)** for PR 27790 at commit [`7f9e801`](https://github.com/apache/spark/commit/7f9e801a6370c161baf1bb6bbb2d364f53c05aa2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594788986 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
SparkQA commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594788322 **[Test build #119326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119326/testReport)** for PR 27790 at commit [`7f9e801`](https://github.com/apache/spark/commit/7f9e801a6370c161baf1bb6bbb2d364f53c05aa2). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594783365 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119332/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594756184 **[Test build #119332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119332/testReport)** for PR 27570 at commit [`2cdc769`](https://github.com/apache/spark/commit/2cdc769cb05f63d5a7e04b66aa950af8bbff4e9b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594783352 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594783352 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594783365 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119332/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594783246 **[Test build #119332 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119332/testReport)** for PR 27570 at commit [`2cdc769`](https://github.com/apache/spark/commit/2cdc769cb05f63d5a7e04b66aa950af8bbff4e9b). * This patch **fails to generate documentation**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
AmplabJenkins removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594781401 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119305/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
AmplabJenkins removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594781386 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
AmplabJenkins commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594781386 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
AmplabJenkins commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594781401 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119305/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
SparkQA removed a comment on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594521263 **[Test build #119305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119305/testReport)** for PR 27366 at commit [`e78bacc`](https://github.com/apache/spark/commit/e78bacc8ce06d0367e5388d82179e4777808284e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource
SparkQA commented on issue #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource URL: https://github.com/apache/spark/pull/27366#issuecomment-594779714 **[Test build #119305 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119305/testReport)** for PR 27366 at commit [`e78bacc`](https://github.com/apache/spark/commit/e78bacc8ce06d0367e5388d82179e4777808284e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594757201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24069/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-594756898 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-594756906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24067/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-594757373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24068/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-594757356 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594757179 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yhuai closed pull request #27536: [SPARK-30784] Use ORC nohive
yhuai closed pull request #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive
yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-594774176 Closing it as we need to use regular orc This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zero323 commented on issue #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API
zero323 commented on issue #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API URL: https://github.com/apache/spark/pull/27331#issuecomment-594760354 > I am okay with adding it in PySpark. Just wanted to make sure if we're going to change DSv2 API shape more or not. If that's expected, let's add them later to reduce dev overhead as new APIs should target 3.1 anyway. If it's expected to not change, I am good with adding it. I don't have strong opinion here, but I guess that one consideration is how much keeping this out of PySpark limits feedback that can gathered and used to drive the evolution of the API. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-594757356 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-594756906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24067/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-594757373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24068/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594757179 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-594756898 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594757201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24069/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-594756210 **[Test build #119331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119331/testReport)** for PR 27571 at commit [`22b9ae5`](https://github.com/apache/spark/commit/22b9ae5b8d8e894dbb3547752e013323888b08ab). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
SparkQA commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-594756239 **[Test build #119330 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119330/testReport)** for PR 27593 at commit [`229aa4c`](https://github.com/apache/spark/commit/229aa4cb284b4c4744ef7443c52f56cd7bc990e1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594756184 **[Test build #119332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119332/testReport)** for PR 27570 at commit [`2cdc769`](https://github.com/apache/spark/commit/2cdc769cb05f63d5a7e04b66aa950af8bbff4e9b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594729588 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594729605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119303/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594729605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119303/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
AmplabJenkins commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594729588 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
AmplabJenkins removed a comment on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594727755 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
AmplabJenkins removed a comment on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594727769 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24066/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function
SparkQA removed a comment on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594500510 **[Test build #119303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119303/testReport)** for PR 27759 at commit [`34d915d`](https://github.com/apache/spark/commit/34d915df0d18b16d41f376dbab6c29ed8cc05bce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
AmplabJenkins commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594727769 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24066/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
AmplabJenkins commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594727755 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function
SparkQA commented on issue #27759: [SPARK-31008][SQL]Support json_array_length function URL: https://github.com/apache/spark/pull/27759#issuecomment-594727575 **[Test build #119303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119303/testReport)** for PR 27759 at commit [`34d915d`](https://github.com/apache/spark/commit/34d915df0d18b16d41f376dbab6c29ed8cc05bce). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
SparkQA commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594726641 **[Test build #119329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119329/testReport)** for PR 27796 at commit [`7576e62`](https://github.com/apache/spark/commit/7576e62837fc97283a7cb1ce06b3835eb63fdad4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
gengliangwang commented on issue #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796#issuecomment-594724554 This PR is to cherry-pick #27763 to branch-3.0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang opened a new pull request #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index
gengliangwang opened a new pull request #27796: [SPARK-31013][CORE][WEBUI] InMemoryStore: improve removeAllByIndexValues over natural key index URL: https://github.com/apache/spark/pull/27796 ### What changes were proposed in this pull request? The method `removeAllByIndexValues` in KVStore is to delete all the objects which have certain values in the given index. However, in the current implementation of `InMemoryStore`, when the given index is the natural key index, there is no special handling for it and a linear search over all the task data is performed. We can improve it by deleting the natural keys directly from the internal hashmap. ### Why are the changes needed? Better performance if the given index for `removeAllByIndexValues` is the natural key index in `InMemoryStore` ### Does this PR introduce any user-facing change? No ### How was this patch tested? Enhance the existing test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode
sarutak commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode URL: https://github.com/apache/spark/pull/27787#issuecomment-594718940 Sometimes application developers develop apps on their laptops and test functionality on it and then, deploy the apps to a cluster. So if resource aware scheduling supports local mode, they can deploy apps without code change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode
jiangxb1987 commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode URL: https://github.com/apache/spark/pull/27787#issuecomment-594717644 @sarutak So you have a use case that is running production workload on local mode? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang edited a comment on issue #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
gengliangwang edited a comment on issue #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27765#issuecomment-594713289 As https://github.com/apache/spark/pull/27716 is also in 3.0 and this PR is the missing part of it, I am merging this one to master and 3.0. However, there is conflict in 3.0 since #27763 is not in it..I will cherry-pick #27763 to 3.0 first. @HeartSaVioR would you cherry-pick this one to 3.0 afterward? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
gengliangwang closed pull request #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27765 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
gengliangwang commented on issue #27765: [SPARK-31014][CORE] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27765#issuecomment-594713289 As https://github.com/apache/spark/pull/27716 is also in 3.0 and this PR is the missing part of it, I am merging this one to master and 3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zsxwing commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
zsxwing commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite URL: https://github.com/apache/spark/pull/27789#issuecomment-594710354 +1 on trying to reproduce it and fix the root issue (or have a confident workaround if that's an issue in an external library). I usually just update `dev/run-tests` to something like: ```shell set -e for i in `seq 0 100`; do build/sbt "sql-kafka-0-10/test-only *KafkaDelegationTokenSuite" done # exec python3 -u ./dev/run-tests.py "$@" ``` and open a PR to run the flaky test 100 (or more) times on Jenkins in one build. If this can reproduce the issue, we can add more logs to investigate further. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
SparkQA commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594709144 **[Test build #119328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119328/testReport)** for PR 27780 at commit [`5ebafd5`](https://github.com/apache/spark/commit/5ebafd532a7c0f20e5088a73f6c73e7b90fccaf3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
AmplabJenkins removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594706075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119311/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
AmplabJenkins removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594706065 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
AmplabJenkins commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594706075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119311/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
AmplabJenkins commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594706065 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
SparkQA removed a comment on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594552830 **[Test build #119311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119311/testReport)** for PR 27637 at commit [`8a2cc85`](https://github.com/apache/spark/commit/8a2cc851e29b1342c408b5534334988b3f8ca523). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594702075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24065/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector
SparkQA commented on issue #27637: [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector URL: https://github.com/apache/spark/pull/27637#issuecomment-594703086 **[Test build #119311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119311/testReport)** for PR 27637 at commit [`8a2cc85`](https://github.com/apache/spark/commit/8a2cc851e29b1342c408b5534334988b3f8ca523). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594702042 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594702042 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594702075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24065/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode
tgravescs commented on issue #27787: [SPARK-31035][WEBUI] Show assigned resource information for local mode URL: https://github.com/apache/spark/pull/27787#issuecomment-594692839 what is the use case for this in local mode? If you are just running on a single box you would generally have visibility into all the resources on the node This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dbtsai commented on issue #27778: [SPARK-31027] [SQL] Refactor DataSourceStrategy to be more extendable
dbtsai commented on issue #27778: [SPARK-31027] [SQL] Refactor DataSourceStrategy to be more extendable URL: https://github.com/apache/spark/pull/27778#issuecomment-594688462 Also merged into 3.0 branch since it's just refactoring without introducing new feature. It will make code maintenance easier for 3.0 and future master. Thanks all for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27785: [SPARK-30934][ML][DOCS] Update ml-guide and ml-migration-guide for 3.0 release
huaxingao commented on issue #27785: [SPARK-30934][ML][DOCS] Update ml-guide and ml-migration-guide for 3.0 release URL: https://github.com/apache/spark/pull/27785#issuecomment-594684927 Thanks @srowen for reviewing! Also cc @WeichenXu123 @zhengruifeng This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
SparkQA removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#issuecomment-594530124 **[Test build #119307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119307/testReport)** for PR 27768 at commit [`2f2e6a9`](https://github.com/apache/spark/commit/2f2e6a9d4257d218f91527601131acdc22a8855c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594677684 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24064/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
AmplabJenkins removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#issuecomment-594663806 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119307/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594677670 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594671399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
SparkQA removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594632158 **[Test build #119320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119320/testReport)** for PR 27772 at commit [`b201c9e`](https://github.com/apache/spark/commit/b201c9e68d7f8c5d3e4145b10f0820c247ca4ad8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins removed a comment on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594671414 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24063/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594671379 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119320/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
AmplabJenkins removed a comment on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#issuecomment-594663789 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins removed a comment on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594671367 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
huaxingao commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-594682827 LGTM @zero323 You may want to delete the dir using ```unlink(modelPath)``` after testing model save/load, and delete the extra blank line near the end of the file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maryannxue commented on a change in pull request #27793: [SPARK-31037][SQL] refine AQE config names
maryannxue commented on a change in pull request #27793: [SPARK-31037][SQL]
refine AQE config names
URL: https://github.com/apache/spark/pull/27793#discussion_r387819088
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -388,99 +397,98 @@ object SQLConf {
.internal()
.doc("Adaptive query execution is skipped when the query does not have
exchanges or " +
"sub-queries. By setting this config to true (together with " +
- s"'${ADAPTIVE_EXECUTION_ENABLED.key}' enabled), Spark will force apply
adaptive query " +
+ s"'${ADAPTIVE_EXECUTION_ENABLED.key}' set to true), Spark will force
apply adaptive query " +
"execution for all supported queries.")
.version("3.0.0")
.booleanConf
.createWithDefault(false)
- val REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.reducePostShufflePartitions")
- .doc(s"When true and '${ADAPTIVE_EXECUTION_ENABLED.key}' is enabled,
this enables reducing " +
-"the number of post-shuffle partitions based on map output
statistics.")
+ val ADVISORY_SHUFFLE_PARTITION_SIZE_IN_BYTES =
+buildConf("spark.sql.adaptive.advisoryShufflePartitionSizeInBytes")
+ .doc("The advisory size in bytes of the shuffle partition during
adaptive optimization. " +
+"It takes effect when Spark coalesces small shuffle partitions or
splits skewed shuffle " +
+"partition.")
.version("3.0.0")
- .booleanConf
- .createWithDefault(true)
+ .fallbackConf(SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE)
- val FETCH_SHUFFLE_BLOCKS_IN_BATCH_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.fetchShuffleBlocksInBatch")
- .doc("Whether to fetch the continuous shuffle blocks in batch. Instead
of fetching blocks " +
-"one by one, fetching continuous shuffle blocks for the same map task
in batch can " +
-"reduce IO and improve performance. Note, multiple continuous blocks
exist in single " +
-s"fetch request only happen when '${ADAPTIVE_EXECUTION_ENABLED.key}'
and " +
-s"'${REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED.key}' is enabled, this
feature also depends " +
-"on a relocatable serializer, the concatenation support codec in use
and the new version " +
-"shuffle fetch protocol.")
+ val COALESCE_SHUFFLE_PARTITIONS_ENABLED =
+buildConf("spark.sql.adaptive.coalesceShufflePartitions.enabled")
Review comment:
`advisoryPartitionSizeInBytes` looks good to me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27792: [SPARK-31038][SQL] Add checkValue for spark.sql.session.timeZone
cloud-fan commented on a change in pull request #27792: [SPARK-31038][SQL] Add
checkValue for spark.sql.session.timeZone
URL: https://github.com/apache/spark/pull/27792#discussion_r387816365
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -1654,6 +1660,8 @@ object SQLConf {
.doc("""The ID of session local timezone, e.g. "GMT",
"America/Los_Angeles", etc.""")
.version("")
.stringConf
+ .checkValue(isValidTimezone, "The time zone should obey rules managed
by" +
+" java.time.zone.ZoneRulesProvider")
Review comment:
This is still unclear, maybe just say: `Cannot resolve the give timezone
with ZoneId.of(_, ZoneId.SHORT_IDS)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594677684 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24064/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594677670 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
SparkQA commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594676463 **[Test build #119327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119327/testReport)** for PR 27772 at commit [`b201c9e`](https://github.com/apache/spark/commit/b201c9e68d7f8c5d3e4145b10f0820c247ca4ad8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27792: [SPARK-31038][SQL] Add checkValue for spark.sql.session.timeZone
cloud-fan commented on a change in pull request #27792: [SPARK-31038][SQL] Add
checkValue for spark.sql.session.timeZone
URL: https://github.com/apache/spark/pull/27792#discussion_r387814355
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -75,6 +77,10 @@ object SQLConf {
}
}
+ def isValidTimezone(zone: String): Boolean = {
Review comment:
can we move it closer to where it's used?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27793: [SPARK-31037][SQL] refine AQE config names
cloud-fan commented on a change in pull request #27793: [SPARK-31037][SQL]
refine AQE config names
URL: https://github.com/apache/spark/pull/27793#discussion_r387813602
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -388,99 +397,98 @@ object SQLConf {
.internal()
.doc("Adaptive query execution is skipped when the query does not have
exchanges or " +
"sub-queries. By setting this config to true (together with " +
- s"'${ADAPTIVE_EXECUTION_ENABLED.key}' enabled), Spark will force apply
adaptive query " +
+ s"'${ADAPTIVE_EXECUTION_ENABLED.key}' set to true), Spark will force
apply adaptive query " +
"execution for all supported queries.")
.version("3.0.0")
.booleanConf
.createWithDefault(false)
- val REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.reducePostShufflePartitions")
- .doc(s"When true and '${ADAPTIVE_EXECUTION_ENABLED.key}' is enabled,
this enables reducing " +
-"the number of post-shuffle partitions based on map output
statistics.")
+ val ADVISORY_SHUFFLE_PARTITION_SIZE_IN_BYTES =
+buildConf("spark.sql.adaptive.advisoryShufflePartitionSizeInBytes")
+ .doc("The advisory size in bytes of the shuffle partition during
adaptive optimization. " +
+"It takes effect when Spark coalesces small shuffle partitions or
splits skewed shuffle " +
+"partition.")
.version("3.0.0")
- .booleanConf
- .createWithDefault(true)
+ .fallbackConf(SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE)
- val FETCH_SHUFFLE_BLOCKS_IN_BATCH_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.fetchShuffleBlocksInBatch")
- .doc("Whether to fetch the continuous shuffle blocks in batch. Instead
of fetching blocks " +
-"one by one, fetching continuous shuffle blocks for the same map task
in batch can " +
-"reduce IO and improve performance. Note, multiple continuous blocks
exist in single " +
-s"fetch request only happen when '${ADAPTIVE_EXECUTION_ENABLED.key}'
and " +
-s"'${REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED.key}' is enabled, this
feature also depends " +
-"on a relocatable serializer, the concatenation support codec in use
and the new version " +
-"shuffle fetch protocol.")
+ val COALESCE_SHUFFLE_PARTITIONS_ENABLED =
+buildConf("spark.sql.adaptive.coalesceShufflePartitions.enabled")
Review comment:
shall we also rename `advisoryShufflePartitionSizeInBytes` to
`advisoryPartitionSizeInBytes`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
cloud-fan commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594673924 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594671379 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119320/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594671399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
AmplabJenkins commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594671414 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24063/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
SparkQA commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594671148 **[Test build #119320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119320/testReport)** for PR 27772 at commit [`b201c9e`](https://github.com/apache/spark/commit/b201c9e68d7f8c5d3e4145b10f0820c247ca4ad8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys
AmplabJenkins commented on issue #27772: [SPARK-31019][SQL] make it clear that people can deduplicate map keys URL: https://github.com/apache/spark/pull/27772#issuecomment-594671367 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
SparkQA commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-594670456 **[Test build #119326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119326/testReport)** for PR 27790 at commit [`7f9e801`](https://github.com/apache/spark/commit/7f9e801a6370c161baf1bb6bbb2d364f53c05aa2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
dongjoon-hyun commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-594669474 Please rebase to the master because the related sub-PR is merged now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27710: [SPARK-30960][SQL] add back the legacy date/timestamp format support in CSV/JSON parser
dongjoon-hyun commented on issue #27710: [SPARK-30960][SQL] add back the legacy date/timestamp format support in CSV/JSON parser URL: https://github.com/apache/spark/pull/27710#issuecomment-594668548 Thank you all! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0
dongjoon-hyun commented on issue #27790: [SPARK-30994][CORE][FOLLOWUP] Add core dep for xerces 2.12.0 URL: https://github.com/apache/spark/pull/27790#issuecomment-59445 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maryannxue commented on a change in pull request #27793: [SPARK-31037][SQL] refine AQE config names
maryannxue commented on a change in pull request #27793: [SPARK-31037][SQL]
refine AQE config names
URL: https://github.com/apache/spark/pull/27793#discussion_r387806656
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -388,99 +397,98 @@ object SQLConf {
.internal()
.doc("Adaptive query execution is skipped when the query does not have
exchanges or " +
"sub-queries. By setting this config to true (together with " +
- s"'${ADAPTIVE_EXECUTION_ENABLED.key}' enabled), Spark will force apply
adaptive query " +
+ s"'${ADAPTIVE_EXECUTION_ENABLED.key}' set to true), Spark will force
apply adaptive query " +
"execution for all supported queries.")
.version("3.0.0")
.booleanConf
.createWithDefault(false)
- val REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.reducePostShufflePartitions")
- .doc(s"When true and '${ADAPTIVE_EXECUTION_ENABLED.key}' is enabled,
this enables reducing " +
-"the number of post-shuffle partitions based on map output
statistics.")
+ val ADVISORY_SHUFFLE_PARTITION_SIZE_IN_BYTES =
+buildConf("spark.sql.adaptive.advisoryShufflePartitionSizeInBytes")
+ .doc("The advisory size in bytes of the shuffle partition during
adaptive optimization. " +
+"It takes effect when Spark coalesces small shuffle partitions or
splits skewed shuffle " +
+"partition.")
.version("3.0.0")
- .booleanConf
- .createWithDefault(true)
+ .fallbackConf(SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE)
- val FETCH_SHUFFLE_BLOCKS_IN_BATCH_ENABLED =
-buildConf("spark.sql.adaptive.shuffle.fetchShuffleBlocksInBatch")
- .doc("Whether to fetch the continuous shuffle blocks in batch. Instead
of fetching blocks " +
-"one by one, fetching continuous shuffle blocks for the same map task
in batch can " +
-"reduce IO and improve performance. Note, multiple continuous blocks
exist in single " +
-s"fetch request only happen when '${ADAPTIVE_EXECUTION_ENABLED.key}'
and " +
-s"'${REDUCE_POST_SHUFFLE_PARTITIONS_ENABLED.key}' is enabled, this
feature also depends " +
-"on a relocatable serializer, the concatenation support codec in use
and the new version " +
-"shuffle fetch protocol.")
+ val COALESCE_SHUFFLE_PARTITIONS_ENABLED =
+buildConf("spark.sql.adaptive.coalesceShufflePartitions.enabled")
Review comment:
I'm fine with "coalescePartitions".
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone
AmplabJenkins commented on issue #27768: [SPARK-31018][CORE] Deprecate support of multiple workers on the same host in Standalone URL: https://github.com/apache/spark/pull/27768#issuecomment-594663806 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119307/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
