[GitHub] [spark] MaxGekk commented on a change in pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
MaxGekk commented on a change in pull request #28824:
URL: https://github.com/apache/spark/pull/28824#discussion_r440628505
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/RebaseDateTimeSuite.scala
##
@@ -416,38 +418,39 @@ class RebaseDateTimeSuite extends SparkFunSuite with
Matchers with SQLHelper {
// clocks were moved backward to become Sunday, 18 November, 1945 01:00:00
AM.
// In this way, the overlap happened w/o Daylight Saving Time.
val hkZid = getZoneId("Asia/Hong_Kong")
+var expected = "1945-11-18 01:30:00.0"
+var ldt = LocalDateTime.of(1945, 11, 18, 1, 30, 0)
+var earlierMicros =
instantToMicros(ldt.atZone(hkZid).withEarlierOffsetAtOverlap().toInstant)
+var laterMicros =
instantToMicros(ldt.atZone(hkZid).withLaterOffsetAtOverlap().toInstant)
+var overlapInterval = MICROS_PER_HOUR
+if (earlierMicros + overlapInterval != laterMicros) {
+ // Old JDK might have an outdated time zone database.
+ // See https://bugs.openjdk.java.net/browse/JDK-8228469: "Hong Kong ...
Its 1945 transition
+ // from JST to HKT was on 11-18 at 02:00, not 09-15 at 00:00"
+ expected = "1945-09-14 23:30:00.0"
+ ldt = LocalDateTime.of(1945, 9, 14, 23, 30, 0)
+ earlierMicros =
instantToMicros(ldt.atZone(hkZid).withEarlierOffsetAtOverlap().toInstant)
+ laterMicros =
instantToMicros(ldt.atZone(hkZid).withLaterOffsetAtOverlap().toInstant)
+ // If time zone db doesn't have overlapping at all, set the overlap
interval to zero.
+ overlapInterval = laterMicros - earlierMicros
+}
+val hkTz = TimeZone.getTimeZone(hkZid)
+val rebasedEarlierMicros = rebaseGregorianToJulianMicros(hkTz,
earlierMicros)
+val rebasedLaterMicros = rebaseGregorianToJulianMicros(hkTz, laterMicros)
+assert(rebasedEarlierMicros + overlapInterval === rebasedLaterMicros)
withDefaultTimeZone(hkZid) {
- var expected = "1945-11-18 01:30:00.0"
- var ldt = LocalDateTime.of(1945, 11, 18, 1, 30, 0)
- var earlierMicros =
instantToMicros(ldt.atZone(hkZid).withEarlierOffsetAtOverlap().toInstant)
- var laterMicros =
instantToMicros(ldt.atZone(hkZid).withLaterOffsetAtOverlap().toInstant)
- var overlapInterval = MICROS_PER_HOUR
- if (earlierMicros + overlapInterval != laterMicros) {
-// Old JDK might have an outdated time zone database.
-// See https://bugs.openjdk.java.net/browse/JDK-8228469: "Hong Kong
... Its 1945 transition
-// from JST to HKT was on 11-18 at 02:00, not 09-15 at 00:00"
-expected = "1945-09-14 23:30:00.0"
-ldt = LocalDateTime.of(1945, 9, 14, 23, 30, 0)
-earlierMicros =
instantToMicros(ldt.atZone(hkZid).withEarlierOffsetAtOverlap().toInstant)
-laterMicros =
instantToMicros(ldt.atZone(hkZid).withLaterOffsetAtOverlap().toInstant)
-// If time zone db doesn't have overlapping at all, set the overlap
interval to zero.
-overlapInterval = laterMicros - earlierMicros
- }
- val rebasedEarlierMicros = rebaseGregorianToJulianMicros(hkZid,
earlierMicros)
- val rebasedLaterMicros = rebaseGregorianToJulianMicros(hkZid,
laterMicros)
def toTsStr(micros: Long): String = toJavaTimestamp(micros).toString
assert(toTsStr(rebasedEarlierMicros) === expected)
assert(toTsStr(rebasedLaterMicros) === expected)
- assert(rebasedEarlierMicros + overlapInterval === rebasedLaterMicros)
// Check optimized rebasing
assert(rebaseGregorianToJulianMicros(earlierMicros) ===
rebasedEarlierMicros)
assert(rebaseGregorianToJulianMicros(laterMicros) === rebasedLaterMicros)
// Check reverse rebasing
assert(rebaseJulianToGregorianMicros(rebasedEarlierMicros) ===
earlierMicros)
assert(rebaseJulianToGregorianMicros(rebasedLaterMicros) === laterMicros)
// Check reverse not-optimized rebasing
Review comment:
No, they don't. Let me move them out of the default JVM time zone.
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644574172 **[Test build #124092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124092/testReport)** for PR 28836 at commit [`154edef`](https://github.com/apache/spark/commit/154edefa188f33aa5d465180a155c9bf34f381dd). * This patch **fails Spark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644527550 **[Test build #124092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124092/testReport)** for PR 28836 at commit [`154edef`](https://github.com/apache/spark/commit/154edefa188f33aa5d465180a155c9bf34f381dd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
SparkQA commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644574854 **[Test build #124103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124103/testReport)** for PR 28824 at commit [`381e950`](https://github.com/apache/spark/commit/381e950d6447238a41af6c766f1c7eae81e585a7). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644574860 **[Test build #124093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
SparkQA commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644574866 **[Test build #124094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124094/testReport)** for PR 28837 at commit [`6a1e8c0`](https://github.com/apache/spark/commit/6a1e8c0c4f861d52a7408d0f72373099a59455c0). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644574855 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types of c
SparkQA commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644574859 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
SparkQA commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644574864 **[Test build #124100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124100/testReport)** for PR 28715 at commit [`a85f511`](https://github.com/apache/spark/commit/a85f51121ede087d6a943275e832ba290caa0966). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644574849 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
SparkQA commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644574870 **[Test build #124084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124084/testReport)** for PR 28746 at commit [`a3ed5c1`](https://github.com/apache/spark/commit/a3ed5c1e213e67b6ba84a2e9b3d487cf766c2704). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644574926 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
gengliangwang commented on a change in pull request #28810:
URL: https://github.com/apache/spark/pull/28810#discussion_r440629435
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -1334,13 +1340,13 @@ object PushPredicateThroughJoin extends
Rule[LogicalPlan] with PredicateHelper {
(rightFilterConditions ++ commonFilterCondition).
reduceLeftOption(And).map(Filter(_, newJoin)).getOrElse(newJoin)
-case FullOuter => f // DO Nothing for Full Outer Join
-case NaturalJoin(_) => sys.error("Untransformed NaturalJoin node")
-case UsingJoin(_, _) => sys.error("Untransformed Using join node")
+
+case jt =>
Review comment:
Nit: jt => other
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PushCNFPredicateThroughJoin.scala
##
@@ -53,9 +60,8 @@ object PushCNFPredicateThroughJoin extends Rule[LogicalPlan]
with PredicateHelpe
Join(newLeft, right, RightOuter, Some(joinCondition), hint)
case LeftOuter | LeftAnti | ExistenceJoin(_) =>
Join(left, newRight, joinType, Some(joinCondition), hint)
- case FullOuter => j
- case NaturalJoin(_) => sys.error("Untransformed NaturalJoin node")
- case UsingJoin(_, _) => sys.error("Untransformed Using join node")
+ case jt =>
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
AmplabJenkins commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644574932 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
SparkQA commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644574869 **[Test build #124104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124104/testReport)** for PR 28810 at commit [`1251b68`](https://github.com/apache/spark/commit/1251b68044611148dcb840d3efa3c39ed9aee18e). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
SparkQA removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644566345 **[Test build #124103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124103/testReport)** for PR 28824 at commit [`381e950`](https://github.com/apache/spark/commit/381e950d6447238a41af6c766f1c7eae81e585a7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
SparkQA removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644566350 **[Test build #124104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124104/testReport)** for PR 28810 at commit [`1251b68`](https://github.com/apache/spark/commit/1251b68044611148dcb840d3efa3c39ed9aee18e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
SparkQA commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644574858 **[Test build #124098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124098/testReport)** for PR 28833 at commit [`479901d`](https://github.com/apache/spark/commit/479901db17d2e79d4a3001ae86f81faa545a5f4a). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644574872 **[Test build #124099 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124099/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644574849 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
SparkQA commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644574873 **[Test build #124101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124101/testReport)** for PR 28838 at commit [`03bf55f`](https://github.com/apache/spark/commit/03bf55fe38a2a70b1b04326fd053c98e82b92d09). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644575068 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array type
AmplabJenkins commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644575049 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644574884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
AmplabJenkins commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644574965 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644574966 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644574984 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
AmplabJenkins removed a comment on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644574965 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
SparkQA removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644514044 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644575068 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644574868 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124092/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644575545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
SparkQA removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644538725 **[Test build #124098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124098/testReport)** for PR 28833 at commit [`479901d`](https://github.com/apache/spark/commit/479901db17d2e79d4a3001ae86f81faa545a5f4a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644574984 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644527552 **[Test build #124093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644574966 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
SparkQA removed a comment on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644560348 **[Test build #124101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124101/testReport)** for PR 28838 at commit [`03bf55f`](https://github.com/apache/spark/commit/03bf55fe38a2a70b1b04326fd053c98e82b92d09). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins removed a comment on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644574926 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
SparkQA removed a comment on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644557745 **[Test build #124100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124100/testReport)** for PR 28715 at commit [`a85f511`](https://github.com/apache/spark/commit/a85f51121ede087d6a943275e832ba290caa0966). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
AmplabJenkins removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644574932 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
SparkQA removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644496861 **[Test build #124084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124084/testReport)** for PR 28746 at commit [`a3ed5c1`](https://github.com/apache/spark/commit/a3ed5c1e213e67b6ba84a2e9b3d487cf766c2704). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644555082 **[Test build #124099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124099/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644575338 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644574884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644497147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
SparkQA removed a comment on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644532968 **[Test build #124094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124094/testReport)** for PR 28837 at commit [`6a1e8c0`](https://github.com/apache/spark/commit/6a1e8c0c4f861d52a7408d0f72373099a59455c0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644575338 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
AmplabJenkins commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644575367 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/ar
AmplabJenkins removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644575049 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644575348 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124093/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins removed a comment on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644574937 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124101/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644574998 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124104/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644575080 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124099/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
AmplabJenkins removed a comment on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644574974 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124100/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins removed a comment on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644575545 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644575374 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124084/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644574994 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124087/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
AmplabJenkins removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644574942 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124103/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644574980 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124098/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array ty
SparkQA removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644536708 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
SparkQA commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644576078 **[Test build #124105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124105/testReport)** for PR 28824 at commit [`9f4f286`](https://github.com/apache/spark/commit/9f4f2863212ec85dcc89de85d65aa9f40217c4be). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644577485 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644576661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
AmplabJenkins removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644576611 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644576873 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
zhengruifeng commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644577984 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins removed a comment on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644575554 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644576661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
sarutak commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644576163 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure
AmplabJenkins commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644576611 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
sarutak commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644575986 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644579384 **[Test build #124109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124109/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
SparkQA commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644579363 **[Test build #124106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124106/testReport)** for PR 28837 at commit [`6a1e8c0`](https://github.com/apache/spark/commit/6a1e8c0c4f861d52a7408d0f72373099a59455c0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644579388 **[Test build #124108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124108/testReport)** for PR 28835 at commit [`9c6ac83`](https://github.com/apache/spark/commit/9c6ac83afbe26f1277a72c04ac3b1ec40a8c7141). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644579440 **[Test build #124110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124110/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644579385 **[Test build #124107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124107/testReport)** for PR 28836 at commit [`c701b32`](https://github.com/apache/spark/commit/c701b321580347c02d22d764a1d4c688beb58bef). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644580130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
maropu commented on a change in pull request #28715:
URL: https://github.com/apache/spark/pull/28715#discussion_r440635513
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala
##
@@ -226,14 +226,19 @@ case class GenerateExec(
"0"
}
val numOutput = metricTerm(ctx, "numOutputRows")
+val requiredInput = {
+ input.zipWithIndex.flatMap { case (ev, i) =>
+if ((parent.inputSet).contains(child.output(i))) Seq(ev) else Seq()
+ }
+} ++ position ++ values
Review comment:
more style nit: how about reformatting it ilke this?
```
val requiredOutputVars = input.zip(child.output).filter(ev =>
parent.inputSet.contains(ev._2))
.map(_._1) ++ position ++ values
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644580084 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644580203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644580203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644580130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
maropu commented on a change in pull request #28715:
URL: https://github.com/apache/spark/pull/28715#discussion_r440636071
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala
##
@@ -51,6 +51,86 @@ class WholeStageCodegenSuite extends QueryTest with
SharedSparkSession
assert(df.collect() === Array(Row(9, 4.5)))
}
+ test("SPARK-31897: GenerateExec should be included in WholeStageCodegen") {
+import testImplicits._
+val arrayData = Seq(("James", Seq("Java", "Scala"), Map("hair" -> "black",
"eye" -> "brown")))
+val df = arrayData.toDF("name", "knownLanguages", "properties")
+
+// Array - explode
+var expDF = df.select($"name", explode($"knownLanguages"), $"properties")
+var plan = expDF.queryExecution.executedPlan
+assert(plan.find {
+ case stage: WholeStageCodegenExec =>
+stage.find(_.isInstanceOf[GenerateExec]).isDefined
+ case _ => false
+}.isDefined)
+checkAnswer(expDF, Array(Row("James", "Java", Map("hair" -> "black", "eye"
-> "brown")),
+ Row("James", "Scala", Map("hair" -> "black", "eye" -> "brown"
+
+
Review comment:
nit : plz remove the one blank.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
maropu commented on a change in pull request #28715:
URL: https://github.com/apache/spark/pull/28715#discussion_r440635929
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala
##
@@ -51,6 +51,86 @@ class WholeStageCodegenSuite extends QueryTest with
SharedSparkSession
assert(df.collect() === Array(Row(9, 4.5)))
}
+ test("SPARK-31897: GenerateExec should be included in WholeStageCodegen") {
Review comment:
nit: please the prefix `SPARK-31897` because this is not a bug.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644580084 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
maropu commented on a change in pull request #28715:
URL: https://github.com/apache/spark/pull/28715#discussion_r440636308
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala
##
@@ -51,6 +51,86 @@ class WholeStageCodegenSuite extends QueryTest with
SharedSparkSession
assert(df.collect() === Array(Row(9, 4.5)))
}
+ test("SPARK-31897: GenerateExec should be included in WholeStageCodegen") {
+import testImplicits._
+val arrayData = Seq(("James", Seq("Java", "Scala"), Map("hair" -> "black",
"eye" -> "brown")))
+val df = arrayData.toDF("name", "knownLanguages", "properties")
+
+// Array - explode
+var expDF = df.select($"name", explode($"knownLanguages"), $"properties")
+var plan = expDF.queryExecution.executedPlan
+assert(plan.find {
+ case stage: WholeStageCodegenExec =>
+stage.find(_.isInstanceOf[GenerateExec]).isDefined
+ case _ => false
+}.isDefined)
+checkAnswer(expDF, Array(Row("James", "Java", Map("hair" -> "black", "eye"
-> "brown")),
+ Row("James", "Scala", Map("hair" -> "black", "eye" -> "brown"
+
+
+// Map - explode
+expDF = df.select($"name", $"knownLanguages", explode($"properties"))
+plan = expDF.queryExecution.executedPlan
+assert(plan.find {
+ case stage: WholeStageCodegenExec =>
+stage.find(_.isInstanceOf[GenerateExec]).isDefined
+ case _ => false
+}.isDefined)
+checkAnswer(expDF,
+ Array(Row("James", List("Java", "Scala"), "hair", "black"),
+Row("James", List("Java", "Scala"), "eye", "brown")))
+
+// Array - posexplode
+expDF = df.select($"name", posexplode($"knownLanguages"))
+plan = expDF.queryExecution.executedPlan
+assert(plan.find {
+ case stage: WholeStageCodegenExec =>
+stage.find(_.isInstanceOf[GenerateExec]).isDefined
+ case _ => false
+}.isDefined)
+checkAnswer(expDF,
+ Array(Row("James", 0, "Java"), Row("James", 1, "Scala")))
+
+// Map - posexplode
+expDF = df.select($"name", posexplode($"properties"))
+plan = expDF.queryExecution.executedPlan
+assert(plan.find {
+ case stage: WholeStageCodegenExec =>
+stage.find(_.isInstanceOf[GenerateExec]).isDefined
+ case _ => false
+}.isDefined)
+checkAnswer(expDF,
+ Array(Row("James", 0, "hair", "black"), Row("James", 1, "eye", "brown")))
+
Review comment:
nit the single blank is enough?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types
HeartSaVioR commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644582455 retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types of c
SparkQA commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644583204 **[Test build #124111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124111/testReport)** for PR 28831 at commit [`d75ad5a`](https://github.com/apache/spark/commit/d75ad5aee7d24f4037b950137d529ed2ed3fb1cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array type
AmplabJenkins commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644583693 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/ar
AmplabJenkins removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644583693 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
cloud-fan commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r440639844 ## File path: sql/hive/src/test/resources/golden/timestamp cast #3-0-732ed232ac592c5e7f7c913a88874fd2 ## @@ -1 +0,0 @@ -1.2 Review comment: It's OK. We have tests for casting double to timestamp in `CastSuite`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
cloud-fan commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r440640021 ## File path: sql/hive/src/test/resources/golden/timestamp cast #3-0-732ed232ac592c5e7f7c913a88874fd2 ## @@ -1 +0,0 @@ -1.2 Review comment: BTW shall we remove this golden file since the test is removed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"
HyukjinKwon commented on pull request #28828: URL: https://github.com/apache/spark/pull/28828#issuecomment-644591847 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"
HyukjinKwon closed pull request #28828: URL: https://github.com/apache/spark/pull/28828 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on a change in pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
GuoPhilipse commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r440651295 ## File path: sql/hive/src/test/resources/golden/timestamp cast #3-0-732ed232ac592c5e7f7c913a88874fd2 ## @@ -1 +0,0 @@ -1.2 Review comment: Ah.i have removed it this early morning just fews minutes after your comment. it has disappeared in last commit.and the following test also has been finished This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"
HeartSaVioR commented on pull request #28828: URL: https://github.com/apache/spark/pull/28828#issuecomment-644609604 Thanks all for reviewing and merging! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
LuciferYang commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644611045 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bart-samwel commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
bart-samwel commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644615299 I'm not a huge fan of this. For one thing, it is not in ANSI SQL 2016. AFAICT it's a Teradata extension that Snowflake adopted as well because they get a lot of Teradata workloads. To be honest, I find it confusing. There's already WHERE and HAVING, and QUALIFY follows HAVING but the evaluation order is different: Syntax: SELECT -> FROM -> WHERE -> GROUP BY -> **HAVING -> QUALIFY** Semantics: FROM -> WHERE -> GROUP BY -> **HAVING -> SELECT -> QUALIFY** And all it does is basically (SELECT * FROM (SELECT -> FROM -> WHERE -> GROUP BY -> HAVING) WHERE ...). To make this work they've invented a third synonym that basically means "filter", but the meaning is less clear than any of the other synonyms because the order in the syntax is right next to HAVING. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
xuanyuanking commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644615113 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
SparkQA commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644616402 **[Test build #124112 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124112/testReport)** for PR 28838 at commit [`03bf55f`](https://github.com/apache/spark/commit/03bf55fe38a2a70b1b04326fd053c98e82b92d09). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
