[GitHub] [spark] HyukjinKwon edited a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
HyukjinKwon edited a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504841543 Actual fix is https://github.com/apache/spark/pull/24946/commits/7e77426f6679b6c32e786a3caedde31d2d574eb7 and https://github.com/apache/spark/pull/24946/commits/485539bd5c0b87f611d031ca7b092f48fd9ec7b5 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cxzl25 commented on a change in pull request #24497: [SPARK-27630][CORE] Properly handle task end events from completed stages
cxzl25 commented on a change in pull request #24497: [SPARK-27630][CORE]
Properly handle task end events from completed stages
URL: https://github.com/apache/spark/pull/24497#discussion_r296574777
##
File path: project/MimaExcludes.scala
##
@@ -79,6 +79,12 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.scheduler.SparkListenerApplicationStart.this"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.scheduler.SparkListenerApplicationStart$"),
+// [SPARK-27630][CORE] Properly handle task end events from completed
stages
Review comment:
I have compared PR title and comment to be consistent.
What comments are missing?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
HyukjinKwon removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504873202 Ah .. `ContinuousCoalesceRDD` is dependent on the thread local not being inherited .. let me see if I can work around. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
SparkQA commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504885441 **[Test build #106822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106822/testReport)** for PR 24946 at commit [`485539b`](https://github.com/apache/spark/commit/485539bd5c0b87f611d031ca7b092f48fd9ec7b5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
SparkQA commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504885737 **[Test build #106819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106819/testReport)** for PR 24947 at commit [`4af96a1`](https://github.com/apache/spark/commit/4af96a1b90ceb54c64c798215d2f5eefe87ad896). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
SparkQA commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504885740 **[Test build #106817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106817/testReport)** for PR 24735 at commit [`73c74df`](https://github.com/apache/spark/commit/73c74df575459d4cc3c8d1ece93d4e2d43e3b7f4). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504885741 **[Test build #106818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106818/testReport)** for PR 24668 at commit [`d9c5903`](https://github.com/apache/spark/commit/d9c59037604763e7cd50a63c59b8f441f35331ee). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
SparkQA commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504885738 **[Test build #106821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106821/testReport)** for PR 24861 at commit [`6345935`](https://github.com/apache/spark/commit/6345935d78e37d934c39fc157ba28a14a7d1d277). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
SparkQA commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504885742 **[Test build #106820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106820/testReport)** for PR 24918 at commit [`9906106`](https://github.com/apache/spark/commit/99061062e48e174978de871c92e663e9be59a4b0). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504885819 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #24497: [SPARK-27630][CORE] Properly handle task end events from completed stages
Ngone51 commented on a change in pull request #24497: [SPARK-27630][CORE]
Properly handle task end events from completed stages
URL: https://github.com/apache/spark/pull/24497#discussion_r296576606
##
File path: project/MimaExcludes.scala
##
@@ -79,6 +79,12 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.scheduler.SparkListenerApplicationStart.this"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.scheduler.SparkListenerApplicationStart$"),
+// [SPARK-27630][CORE] Properly handle task end events from completed
stages
Review comment:
Oops, well enough.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504885825 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106820/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504885990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106818/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
AmplabJenkins commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504885831 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106821/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504885985 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504885839 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106819/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504885836 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
AmplabJenkins commented on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504885824 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
SparkQA removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504874979 **[Test build #106820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106820/testReport)** for PR 24918 at commit [`9906106`](https://github.com/apache/spark/commit/99061062e48e174978de871c92e663e9be59a4b0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504885836 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
SparkQA removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504878924 **[Test build #106821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106821/testReport)** for PR 24861 at commit [`6345935`](https://github.com/apache/spark/commit/6345935d78e37d934c39fc157ba28a14a7d1d277). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504885819 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
SparkQA removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504873174 **[Test build #106819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106819/testReport)** for PR 24947 at commit [`4af96a1`](https://github.com/apache/spark/commit/4af96a1b90ceb54c64c798215d2f5eefe87ad896). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504885985 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
SparkQA removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504855738 **[Test build #106818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106818/testReport)** for PR 24668 at commit [`d9c5903`](https://github.com/apache/spark/commit/d9c59037604763e7cd50a63c59b8f441f35331ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
AmplabJenkins commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504886324 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
AmplabJenkins commented on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504886328 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106817/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
AmplabJenkins removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504885824 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
SparkQA removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504850736 **[Test build #106817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106817/testReport)** for PR 24735 at commit [`73c74df`](https://github.com/apache/spark/commit/73c74df575459d4cc3c8d1ece93d4e2d43e3b7f4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
AmplabJenkins removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504886324 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504885990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106818/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504885839 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106819/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
AmplabJenkins removed a comment on issue #24861: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG URL: https://github.com/apache/spark/pull/24861#issuecomment-504885831 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106821/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504885825 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106820/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
AmplabJenkins removed a comment on issue #24735: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs URL: https://github.com/apache/spark/pull/24735#issuecomment-504886328 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106817/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
AmplabJenkins commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504887247 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12039/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
AmplabJenkins commented on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504887239 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504887299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12040/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504887291 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
yaooqinn commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504887131 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
AmplabJenkins removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504887239 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504887291 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)
AmplabJenkins removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-504887247 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12039/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
AmplabJenkins removed a comment on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504887299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12040/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
SparkQA commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504887731 **[Test build #106823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106823/testReport)** for PR 24947 at commit [`4af96a1`](https://github.com/apache/spark/commit/4af96a1b90ceb54c64c798215d2f5eefe87ad896). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
SparkQA commented on issue #24918: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function. URL: https://github.com/apache/spark/pull/24918#issuecomment-504887760 **[Test build #106824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106824/testReport)** for PR 24918 at commit [`10f5d77`](https://github.com/apache/spark/commit/10f5d77f740a24d168810f7ba876d4d52e367414). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mgaido91 commented on issue #24940: [SPARK-28135] Better double support in SQL ceil/floor functions
mgaido91 commented on issue #24940: [SPARK-28135] Better double support in SQL ceil/floor functions URL: https://github.com/apache/spark/pull/24940#issuecomment-504888907 I think Spark should not be bound completely to Hive's behavior. Most of other SQL engines work properly in this case and it is a correctness issue. My main doubts regard the behavior change which the PR introduces: the return type of the function is different and this may break running workloads. Since the next release is going to be 3.0, it may be considered fine, but at very least we need to add a statement in the migration guide. Moreover we would also need a test in order to enforce the behavior in those cases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504889512 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12041/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins commented on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504889504 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504889512 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12041/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
AmplabJenkins removed a comment on issue #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception URL: https://github.com/apache/spark/pull/24947#issuecomment-504889504 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24947: [SPARK-28143][SQL]IN expression missing attribute should throw analysis exception
HyukjinKwon commented on a change in pull request #24947: [SPARK-28143][SQL]IN
expression missing attribute should throw analysis exception
URL: https://github.com/apache/spark/pull/24947#discussion_r296583009
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
##
@@ -581,7 +581,6 @@ object FunctionRegistry {
val params = Seq.fill(expressions.size)(classOf[Expression])
val f = constructors.find(_.getParameterTypes.toSeq ==
params).getOrElse {
val validParametersCount = constructors
-.filter(_.getParameterTypes.forall(_ == classOf[Expression]))
Review comment:
This partially reverts #21226. Previous logic was to detect SQL related
expressions only but now it includes all constructors that are not related to
SQL expressions. Can you elaborate why we revert?
Looks we should rather specially handle `In`'s constructors or handle when
`validParametersCount` is empty.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
HyukjinKwon commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504893810 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504894271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12042/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504894264 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504894264 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504894271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12042/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-504894929 **[Test build #106825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106825/testReport)** for PR 24668 at commit [`d9c5903`](https://github.com/apache/spark/commit/d9c59037604763e7cd50a63c59b8f441f35331ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504899028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12043/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r296589966
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleClient.java
##
@@ -97,15 +97,16 @@ public void fetchBlocks(
String execId,
String[] blockIds,
BlockFetchingListener listener,
- DownloadFileManager downloadFileManager) {
+ DownloadFileManager downloadFileManager,
+ int shuffleGenerationId) {
Review comment:
Copy that, done in 96bab1e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r296589853
##
File path: core/src/main/scala/org/apache/spark/network/BlockDataManager.scala
##
@@ -30,7 +30,7 @@ trait BlockDataManager {
* Interface to get local block data. Throws an exception if the block
cannot be found or
* cannot be read successfully.
*/
- def getBlockData(blockId: BlockId): ManagedBuffer
+ def getBlockData(blockId: BlockId, shuffleGenerationId: Int): ManagedBuffer
Review comment:
Thanks for the suggestion, done this in 96bab1e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504899023 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r296590043
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java
##
@@ -78,7 +78,8 @@ public OneForOneBlockFetcher(
String[] blockIds,
BlockFetchingListener listener,
TransportConf transportConf,
- DownloadFileManager downloadFileManager) {
+ DownloadFileManager downloadFileManager,
+ int shuffleGenerationId) {
Review comment:
Thanks, done in 96bab1e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504899023 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504899028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12043/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r296590257 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java ## @@ -83,6 +84,7 @@ private final int mapId; private final Serializer serializer; private final IndexShuffleBlockResolver shuffleBlockResolver; + private final int shuffleGenerationId; Review comment: Thanks, done in 96bab1e. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r296590128
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java
##
@@ -110,7 +111,7 @@ private boolean isShuffleBlocks(String[] blockIds) {
* org.apache.spark.MapOutputTracker.convertMapStatuses.
*/
private FetchShuffleBlocks createFetchShuffleBlocksMsg(
- String appId, String execId, String[] blockIds) {
+ String appId, String execId, String[] blockIds, int shuffleGenerationId)
{
Review comment:
Done in 96bab1e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r296590187 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleClient.java ## @@ -41,14 +41,16 @@ *If it's not null, the remote blocks will be streamed *into temp shuffle files to reduce the memory usage, otherwise, *they will be kept in memory. + * @param shuffleGenerationId the shuffle generation id for all block ids to fetch. */ public abstract void fetchBlocks( String host, int port, String execId, String[] blockIds, BlockFetchingListener listener, - DownloadFileManager downloadFileManager); + DownloadFileManager downloadFileManager, + int shuffleGenerationId); Review comment: Done in 96bab1e. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r296590371
##
File path:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala
##
@@ -51,26 +51,35 @@ private[spark] class IndexShuffleBlockResolver(
private val transportConf = SparkTransportConf.fromSparkConf(conf, "shuffle")
- def getDataFile(shuffleId: Int, mapId: Int): File = {
-blockManager.diskBlockManager.getFile(ShuffleDataBlockId(shuffleId, mapId,
NOOP_REDUCE_ID))
+ def getDataFile(
+ shuffleId: Int,
+ mapId: Int,
+ shuffleGenerationId: Int = -1): File = {
Review comment:
Thanks, done in 96bab1e and remove the default value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r296590430 ## File path: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala ## @@ -75,9 +77,10 @@ private[spark] class SortShuffleManager(conf: SparkConf) extends ShuffleManager } /** - * A mapping from shuffle ids to the number of mappers producing output for those shuffles. + * A mapping from shuffle ids to the tuple of number of mappers producing output and + * indeterminate stage attempt id for those shuffles. Review comment: Thanks, done in 96bab1e. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
SparkQA commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504899713 **[Test build #106826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106826/testReport)** for PR 24892 at commit [`96bab1e`](https://github.com/apache/spark/commit/96bab1e61d0a6c65ade1900708ced998abf9f8ca). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum opened a new pull request #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
wangyum opened a new pull request #24948: [SPARK-28093][SQL][FOLLOW-UP] Update
trim function behavior changes to migration guide
URL: https://github.com/apache/spark/pull/24948
## What changes were proposed in this pull request?
We changed our non-standard syntax for `trim` function in #24902 from
`TRIM(trimStr, str)` to `TRIM(str, trimStr)` to be compatible with other
databases. This pr update the migration guide.
I checked various databases(PostgreSQL, Teradata, Vertica, Oracle, DB2, SQL
Server 2019, MySQL, Hive, Presto) and it seems that only PostgreSQL and Presto
support this non-standard syntax.
**PostgreSQL**:
```sql
postgres=# select substr(version(), 0, 16), trim('yxTomxx', 'x');
substr | btrim
-+---
PostgreSQL 11.3 | yxTom
(1 row)
```
**Presto**:
```sql
presto> select trim('yxTomxx', 'x');
_col0
---
yxTom
(1 row)
```
## How was this patch tested?
manual tests
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] VinceShieh commented on issue #19318: [WIP][SPARK-22096][ML] use aggregateByKeyLocally in feature frequency calc…
VinceShieh commented on issue #19318: [WIP][SPARK-22096][ML] use aggregateByKeyLocally in feature frequency calc… URL: https://github.com/apache/spark/pull/19318#issuecomment-504903730 it seems 'aggregateByKeyLocally' is not implemented This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
AmplabJenkins commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide URL: https://github.com/apache/spark/pull/24948#issuecomment-504904074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12044/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
AmplabJenkins commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide URL: https://github.com/apache/spark/pull/24948#issuecomment-504904062 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] peter-toth opened a new pull request #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
peter-toth opened a new pull request #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949 ## What changes were proposed in this pull request? This PR adds some more WITH test cases as a follow-up to https://github.com/apache/spark/pull/24842 ## How was this patch tested? Add new UTs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504904625 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
SparkQA commented on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide URL: https://github.com/apache/spark/pull/24948#issuecomment-504904743 **[Test build #106827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106827/testReport)** for PR 24948 at commit [`ae2c16e`](https://github.com/apache/spark/commit/ae2c16e0425053af0e710b28deea6801ed2911a5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
AmplabJenkins removed a comment on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide URL: https://github.com/apache/spark/pull/24948#issuecomment-504904062 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide
AmplabJenkins removed a comment on issue #24948: [SPARK-28093][SQL][FOLLOW-UP] Update trim function behavior changes to migration guide URL: https://github.com/apache/spark/pull/24948#issuecomment-504904074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12044/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] peter-toth commented on issue #24842: [SPARK-28002][SQL] Support WITH clause column aliases
peter-toth commented on issue #24842: [SPARK-28002][SQL] Support WITH clause column aliases URL: https://github.com/apache/spark/pull/24842#issuecomment-504905744 @gatorsmile @dongjoon-hyun I opened https://github.com/apache/spark/pull/24949 to add some new test cases. Please let me know if you want more cases. Please note that I'm also working on https://github.com/apache/spark/pull/24860 and it will add many new tests that cover WITH column aliases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504904625 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504906200 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504906425 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504906200 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504906589 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504906598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12045/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504906598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12045/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
AmplabJenkins removed a comment on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504906589 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files
SparkQA commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-504907241 **[Test build #106828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106828/testReport)** for PR 24892 at commit [`721f0d5`](https://github.com/apache/spark/commit/721f0d566ac6576070acf34017a5f46a46ea88d6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN]
Remove unnecessary log from SparkRackResolver
URL: https://github.com/apache/spark/pull/24935#discussion_r296600480
##
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
##
@@ -72,8 +72,6 @@ private[spark] class SparkRackResolver(conf: Configuration)
extends Logging {
val rNameList = dnsToSwitchMapping.resolve(hostNames.toList.asJava).asScala
if (rNameList == null || rNameList.isEmpty) {
hostNames.foreach(nodes += new NodeBase(_, NetworkTopology.DEFAULT_RACK))
- logInfo(s"Got an error when resolving hostNames. " +
Review comment:
Hmm, misunderstood the intention. I've made the suggested change.
@HeartSaVioR this case I don't think early return needed (not considering
this case, yeah I'm one of the guys who's not fan of multiple return paths).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
HyukjinKwon commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504909014 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
HyukjinKwon commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504909051 add to whitelist This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins removed a comment on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504906425 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24935: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
SparkQA commented on issue #24935: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver URL: https://github.com/apache/spark/pull/24935#issuecomment-504909760 **[Test build #106830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106830/testReport)** for PR 24935 at commit [`3b7bf80`](https://github.com/apache/spark/commit/3b7bf80038910ea76a26ae2a10c973650987be06). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN]
Remove unnecessary log from SparkRackResolver
URL: https://github.com/apache/spark/pull/24935#discussion_r296603267
##
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
##
@@ -72,8 +72,6 @@ private[spark] class SparkRackResolver(conf: Configuration)
extends Logging {
val rNameList = dnsToSwitchMapping.resolve(hostNames.toList.asJava).asScala
if (rNameList == null || rNameList.isEmpty) {
hostNames.foreach(nodes += new NodeBase(_, NetworkTopology.DEFAULT_RACK))
- logInfo(s"Got an error when resolving hostNames. " +
Review comment:
Wanted to highlight the intention with `if (rNameList.nonEmpty)`. It can be
dropped if you think since zip with empty list remains an empty collection.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
SparkQA commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504909758 **[Test build #106829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106829/testReport)** for PR 24949 at commit [`efc96bb`](https://github.com/apache/spark/commit/efc96bb3b6dd4fff16249f377866b485b7cf89a1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN]
Remove unnecessary log from SparkRackResolver
URL: https://github.com/apache/spark/pull/24935#discussion_r296603267
##
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
##
@@ -72,8 +72,6 @@ private[spark] class SparkRackResolver(conf: Configuration)
extends Logging {
val rNameList = dnsToSwitchMapping.resolve(hostNames.toList.asJava).asScala
if (rNameList == null || rNameList.isEmpty) {
hostNames.foreach(nodes += new NodeBase(_, NetworkTopology.DEFAULT_RACK))
- logInfo(s"Got an error when resolving hostNames. " +
Review comment:
Wanted to highlight the intention with `if (rNameList.nonEmpty)`. It can be
dropped if you think, since zip with empty list remains an empty collection.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504911759 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12046/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases
AmplabJenkins commented on issue #24949: [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases URL: https://github.com/apache/spark/pull/24949#issuecomment-504911753 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
