[GitHub] [spark] AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values
AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529191074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15302/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values
AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529191074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15302/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values
AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529191073 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25708: [SPARK-28141][SQL] Support special date values
SparkQA commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529190677 **[Test build #110307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110307/testReport)** for PR 25708 at commit [`dabb6ec`](https://github.com/apache/spark/commit/dabb6ec3dca66dc1a77f38996b24419c63cfcea2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL] Support special date values
MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL]
Support special date values
URL: https://github.com/apache/spark/pull/25708#discussion_r322002594
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -848,4 +852,40 @@ object DateTimeUtils {
val sinceEpoch = BigDecimal(timestamp) / MICROS_PER_SECOND + offset
new Decimal().set(sinceEpoch, 20, 6)
}
+
+ def currentDate(zoneId: ZoneId): SQLDate =
localDateToDays(LocalDate.now(zoneId))
+
+ private val specialDate =
"""(EPOCH|NOW|TODAY|TOMORROW|YESTERDAY)\p{Blank}*(.*)""".r
+ /**
+ * Converts notational shorthands that are converted to ordinary dates.
+ * @param input - left trimmed string
+ * @param zoneId - zone identifier used to get the current date.
+ * @return some of days since the epoch if the conversion completed
successfully otherwise None.
+ */
+ def convertSpecialDate(input: String, zoneId: ZoneId): Option[SQLDate] = {
+def isValidZoneId(z: String): Boolean = {
Review comment:
Changed the pattern to cut off blanks at the end. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529189551 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529189552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110302/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529189551 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529189552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110302/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL] Support special date values
MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL]
Support special date values
URL: https://github.com/apache/spark/pull/25708#discussion_r322001930
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -848,4 +852,40 @@ object DateTimeUtils {
val sinceEpoch = BigDecimal(timestamp) / MICROS_PER_SECOND + offset
new Decimal().set(sinceEpoch, 20, 6)
}
+
+ def currentDate(zoneId: ZoneId): SQLDate =
localDateToDays(LocalDate.now(zoneId))
+
+ private val specialDate =
"""(EPOCH|NOW|TODAY|TOMORROW|YESTERDAY)\p{Blank}*(.*)""".r
+ /**
Review comment:
Added and renamed `specialDate` to `specialValue` since it is applicable to
other types as well
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
SparkQA removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529179673 **[Test build #110302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110302/testReport)** for PR 25690 at commit [`b3613c0`](https://github.com/apache/spark/commit/b3613c077862ba58b15d49b4cd33d87f62384c2b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529189434 **[Test build #110302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110302/testReport)** for PR 25690 at commit [`b3613c0`](https://github.com/apache/spark/commit/b3613c077862ba58b15d49b4cd33d87f62384c2b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL] Support special date values
MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL]
Support special date values
URL: https://github.com/apache/spark/pull/25708#discussion_r322001846
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -848,4 +852,40 @@ object DateTimeUtils {
val sinceEpoch = BigDecimal(timestamp) / MICROS_PER_SECOND + offset
new Decimal().set(sinceEpoch, 20, 6)
}
+
+ def currentDate(zoneId: ZoneId): SQLDate =
localDateToDays(LocalDate.now(zoneId))
+
+ private val specialDate =
"""(EPOCH|NOW|TODAY|TOMORROW|YESTERDAY)\p{Blank}*(.*)""".r
+ /**
+ * Converts notational shorthands that are converted to ordinary dates.
+ * @param input - left trimmed string
+ * @param zoneId - zone identifier used to get the current date.
+ * @return some of days since the epoch if the conversion completed
successfully otherwise None.
+ */
+ def convertSpecialDate(input: String, zoneId: ZoneId): Option[SQLDate] = {
+def isValidZoneId(z: String): Boolean = {
+ z == "" || Try { getZoneId(z) }.isSuccess
Review comment:
I don't think the reasons from the style guide are applicable here. And code
looks slightly ugly. Maybe you have better implementation than:
```scala
def isValidZoneId(z: String): Boolean = {
if (z == "") return true
try {
getZoneId(z)
true
} catch {
case NonFatal(_) => false
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529187604 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110306/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
SparkQA removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529186082 **[Test build #110306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110306/testReport)** for PR 25694 at commit [`068352e`](https://github.com/apache/spark/commit/068352e57ca4a3ec3aa65841b26d5a5a1aad431f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529187602 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529187602 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529187604 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110306/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
SparkQA commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529187576 **[Test build #110306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110306/testReport)** for PR 25694 at commit [`068352e`](https://github.com/apache/spark/commit/068352e57ca4a3ec3aa65841b26d5a5a1aad431f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cxzl25 commented on issue #23516: [SPARK-26598] Fix HiveThriftServer2 set hiveconf and hivevar in every sql
cxzl25 commented on issue #23516: [SPARK-26598] Fix HiveThriftServer2 set hiveconf and hivevar in every sql URL: https://github.com/apache/spark/pull/23516#issuecomment-529187125 @wangyum ok, I will submit pr later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on issue #23516: [SPARK-26598] Fix HiveThriftServer2 set hiveconf and hivevar in every sql
wangyum commented on issue #23516: [SPARK-26598] Fix HiveThriftServer2 set hiveconf and hivevar in every sql URL: https://github.com/apache/spark/pull/23516#issuecomment-529186770 @cxzl25 Could you take this over? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r32255
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -40,23 +40,47 @@ class EquivalentExpressions {
override def hashCode: Int = e.semanticHash()
}
+ /**
+ * Wrapper around an Expression that provides structural semantic equality.
+ */
+ case class StructuralExpr(e: Expression) {
+def normalized(expr: Expression): Expression = {
+ expr.transformUp {
+case b: ParameterizedBoundReference =>
+ b.copy(parameter = "")
+ }
+}
+override def equals(o: Any): Boolean = o match {
+ case other: StructuralExpr =>
+normalized(e).semanticEquals(normalized(other.e))
+ case _ => false
+}
+
+override def hashCode: Int = normalized(e).semanticHash()
+ }
+
+ type EquivalenceMap = mutable.HashMap[Expr, mutable.ArrayBuffer[Expression]]
+
// For each expression, the set of equivalent expressions.
private val equivalenceMap = mutable.HashMap.empty[Expr,
mutable.ArrayBuffer[Expression]]
+ // For each expression, the set of structurally equivalent expressions.
+ private val structEquivalenceMap = mutable.HashMap.empty[StructuralExpr,
EquivalenceMap]
Review comment:
Naturally thinking from the comment, I thought the type's
`mutable.HashMap.empty[StructuralExpr, mutable.ArrayBuffer[Expression]]`. Is
the value `EquivalenceMap` because you reuse the existing logic?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
SparkQA commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529186082 **[Test build #110306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110306/testReport)** for PR 25694 at commit [`068352e`](https://github.com/apache/spark/commit/068352e57ca4a3ec3aa65841b26d5a5a1aad431f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529185966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15301/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529185964 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529185966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15301/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529185964 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-529185686 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
AmplabJenkins removed a comment on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-528320009 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25708: [SPARK-28141][SQL] Support special date values
maropu commented on a change in pull request #25708: [SPARK-28141][SQL] Support
special date values
URL: https://github.com/apache/spark/pull/25708#discussion_r321999711
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -848,4 +852,40 @@ object DateTimeUtils {
val sinceEpoch = BigDecimal(timestamp) / MICROS_PER_SECOND + offset
new Decimal().set(sinceEpoch, 20, 6)
}
+
+ def currentDate(zoneId: ZoneId): SQLDate =
localDateToDays(LocalDate.now(zoneId))
+
+ private val specialDate =
"""(EPOCH|NOW|TODAY|TOMORROW|YESTERDAY)\p{Blank}*(.*)""".r
+ /**
+ * Converts notational shorthands that are converted to ordinary dates.
+ * @param input - left trimmed string
+ * @param zoneId - zone identifier used to get the current date.
+ * @return some of days since the epoch if the conversion completed
successfully otherwise None.
+ */
+ def convertSpecialDate(input: String, zoneId: ZoneId): Option[SQLDate] = {
+def isValidZoneId(z: String): Boolean = {
+ z == "" || Try { getZoneId(z) }.isSuccess
Review comment:
Just for the codestyle?
https://github.com/databricks/scala-style-guide#exception
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321999396
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -65,35 +89,102 @@ class EquivalentExpressions {
}
/**
- * Adds the expression to this data structure recursively. Stops if a
matching expression
- * is found. That is, if `expr` has already been added, its children are not
added.
+ * Adds each expression to structural expression data structure, grouping
them with existing
+ * structurally equivalent expressions. Non-recursive.
+ */
+ def addStructExpr(ctx: CodegenContext, expr: Expression): Unit = {
+if (expr.deterministic) {
+ val refs = expr.collect {
+case b: BoundReference => b
+ }
+
+ // For structural equivalent expressions, we need to pass in int type
ordinals into
+ // split functions. If the number of ordinals is more than JVM function
limit, we skip
+ // this expression.
+ // We calculate function parameter length by the number of ints plus
`INPUT_ROW` plus
+ // a int type result array index.
+ val parameterLength = CodeGenerator.calculateParamLength(refs.map(_ =>
Literal(0))) + 2
+ if (CodeGenerator.isValidParamLength(parameterLength)) {
Review comment:
If the length goes over the limit, the current logic gives up eliminating
common exprs? If so, can we fall back into the non-structural mode?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clien
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529184637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110300/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529184637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110300/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
SparkQA removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178610 **[Test build #110300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110300/testReport)** for PR 25720 at commit [`6051f0b`](https://github.com/apache/spark/commit/6051f0ba2c4e39effd55eaabe4d1799f4ecf84e0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clien
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529184635 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529184635 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529184602 **[Test build #110300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110300/testReport)** for PR 25720 at commit [`6051f0b`](https://github.com/apache/spark/commit/6051f0ba2c4e39effd55eaabe4d1799f4ecf84e0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL] Support special date values
MaxGekk commented on a change in pull request #25708: [SPARK-28141][SQL]
Support special date values
URL: https://github.com/apache/spark/pull/25708#discussion_r321999063
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -848,4 +852,40 @@ object DateTimeUtils {
val sinceEpoch = BigDecimal(timestamp) / MICROS_PER_SECOND + offset
new Decimal().set(sinceEpoch, 20, 6)
}
+
+ def currentDate(zoneId: ZoneId): SQLDate =
localDateToDays(LocalDate.now(zoneId))
+
+ private val specialDate =
"""(EPOCH|NOW|TODAY|TOMORROW|YESTERDAY)\p{Blank}*(.*)""".r
+ /**
+ * Converts notational shorthands that are converted to ordinary dates.
+ * @param input - left trimmed string
+ * @param zoneId - zone identifier used to get the current date.
+ * @return some of days since the epoch if the conversion completed
successfully otherwise None.
+ */
+ def convertSpecialDate(input: String, zoneId: ZoneId): Option[SQLDate] = {
+def isValidZoneId(z: String): Boolean = {
+ z == "" || Try { getZoneId(z) }.isSuccess
Review comment:
Why?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #25708: [SPARK-28141][SQL] Support special date values
MaxGekk commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529184302 @maropu Thank you for your review. > You dropped inf/-inf in the supported? I created the separate JIRA https://issues.apache.org/jira/browse/SPARK-29006 because the feature requires special date/timestamp values and handle them in all date/timestamp functions in the special way. > How about the timestamp case? Here is the JIRA https://issues.apache.org/jira/browse/SPARK-29012 and the PR: https://github.com/apache/spark/pull/25716 . I am still working on it because when I uncommented tests in `timestamp.sql`, it stated failing, see https://github.com/apache/spark/pull/25716#issuecomment-529013241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321998868
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -65,35 +89,102 @@ class EquivalentExpressions {
}
/**
- * Adds the expression to this data structure recursively. Stops if a
matching expression
- * is found. That is, if `expr` has already been added, its children are not
added.
+ * Adds each expression to structural expression data structure, grouping
them with existing
+ * structurally equivalent expressions. Non-recursive.
+ */
+ def addStructExpr(ctx: CodegenContext, expr: Expression): Unit = {
+if (expr.deterministic) {
+ val refs = expr.collect {
+case b: BoundReference => b
Review comment:
nit: `case b: BoundReference => Literal(0)`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs
SparkQA commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs URL: https://github.com/apache/spark/pull/24981#issuecomment-529183525 **[Test build #110305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110305/testReport)** for PR 24981 at commit [`51dcbdc`](https://github.com/apache/spark/commit/51dcbdc00bd4c795293428585cd5fd3bc0665f27). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs
AmplabJenkins removed a comment on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs URL: https://github.com/apache/spark/pull/24981#issuecomment-529183384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15300/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529183383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15299/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529183381 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321998484
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala
##
@@ -65,6 +65,40 @@ case class BoundReference(ordinal: Int, dataType: DataType,
nullable: Boolean)
}
}
+/**
+ * This bound reference points to a parameterized slot in an input tuple. It
is used in
+ * common sub-expression elimination. When some common sub-expressions have
same structural
+ * but different slots of input tuple, we replace `BoundReference` with this
parameterized
+ * version. The slot position is parameterized and is given at runtime.
+ */
+case class ParameterizedBoundReference(parameter: String, dataType: DataType,
nullable: Boolean)
+ extends LeafExpression {
+
+ override def toString: String = s"input[$parameter,
${dataType.simpleString}, $nullable]"
+
+ override def eval(input: InternalRow): Any = {
+throw new UnsupportedOperationException(
+ "ParameterizedBoundReference does not implement eval")
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
+assert(ctx.currentVars == null && ctx.INPUT_ROW != null,
+ "ParameterizedBoundReference can not be used in whole-stage codegen
yet.")
Review comment:
Any barrier to support the whole-stage codegen case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs
AmplabJenkins removed a comment on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs URL: https://github.com/apache/spark/pull/24981#issuecomment-529183382 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529183383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15299/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs
AmplabJenkins commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs URL: https://github.com/apache/spark/pull/24981#issuecomment-529183384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15300/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs
AmplabJenkins commented on issue #24981: [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs URL: https://github.com/apache/spark/pull/24981#issuecomment-529183382 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529183381 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529183008 **[Test build #110304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110304/testReport)** for PR 25690 at commit [`c4d6380`](https://github.com/apache/spark/commit/c4d638014d7a7124dc2de229d9735685e5b242d0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529182430 **[Test build #110303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110303/testReport)** for PR 25690 at commit [`833fd4f`](https://github.com/apache/spark/commit/833fd4f3ac51731a2dbebbdfd19ae5f157d78d9e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529182294 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529182295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15298/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529182295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15298/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529182294 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r321996545 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala ## @@ -65,6 +65,40 @@ case class BoundReference(ordinal: Int, dataType: DataType, nullable: Boolean) } } +/** + * This bound reference points to a parameterized slot in an input tuple. It is used in + * common sub-expression elimination. When some common sub-expressions have same structural + * but different slots of input tuple, we replace `BoundReference` with this parameterized + * version. The slot position is parameterized and is given at runtime. + */ +case class ParameterizedBoundReference(parameter: String, dataType: DataType, nullable: Boolean) Review comment: nit: `parameter` -> `variableNameForOrdinal` or `paramNameForOrdinal`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321996734
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -40,23 +40,47 @@ class EquivalentExpressions {
override def hashCode: Int = e.semanticHash()
}
+ /**
+ * Wrapper around an Expression that provides structural semantic equality.
+ */
+ case class StructuralExpr(e: Expression) {
+def normalized(expr: Expression): Expression = {
+ expr.transformUp {
+case b: ParameterizedBoundReference =>
Review comment:
To avoid unnecessary plan copys, we can check this equality based on
`BoundReference` (by just copying it like `b.copy(ordinal = 0 or -1?)`)? IIUC
its ok to replace `BoundReference` with `ParameterizedBoundReference` just when
generating code in
https://github.com/apache/spark/pull/25717/files#diff-8bcc5aea39c73d4bf38aef6f6951d42cR1117?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321996616
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -65,35 +89,102 @@ class EquivalentExpressions {
}
/**
- * Adds the expression to this data structure recursively. Stops if a
matching expression
- * is found. That is, if `expr` has already been added, its children are not
added.
+ * Adds each expression to structural expression data structure, grouping
them with existing
+ * structurally equivalent expressions. Non-recursive.
+ */
+ def addStructExpr(ctx: CodegenContext, expr: Expression): Unit = {
+if (expr.deterministic) {
+ val refs = expr.collect {
+case b: BoundReference => b
+ }
+
+ // For structural equivalent expressions, we need to pass in int type
ordinals into
+ // split functions. If the number of ordinals is more than JVM function
limit, we skip
+ // this expression.
+ // We calculate function parameter length by the number of ints plus
`INPUT_ROW` plus
+ // a int type result array index.
+ val parameterLength = CodeGenerator.calculateParamLength(refs.map(_ =>
Literal(0))) + 2
+ if (CodeGenerator.isValidParamLength(parameterLength)) {
+val parameterizedExpr = parameterizedBoundReferences(ctx, expr)
+
+val e: StructuralExpr = StructuralExpr(parameterizedExpr)
+val f = structEquivalenceMap.get(e)
+if (f.isDefined) {
+ addExpr(expr, f.get)
+} else {
+ val exprMap = mutable.HashMap.empty[Expr,
mutable.ArrayBuffer[Expression]]
+ addExpr(expr, exprMap)
+ structEquivalenceMap.put(e, exprMap)
+}
+ }
+}
+ }
+
+ /**
+ * Replaces bound references in given expression by parameterized bound
references.
*/
- def addExprTree(expr: Expression): Unit = {
-val skip = expr.isInstanceOf[LeafExpression] ||
+ private def parameterizedBoundReferences(ctx: CodegenContext, expr:
Expression): Expression = {
+expr.transformUp {
+ case b: BoundReference =>
+val param = ctx.freshName("boundInput")
+ParameterizedBoundReference(param, b.dataType, b.nullable)
+}
+ }
+
+ /**
+ * Checks if we skip add sub-expressions for given expression.
+ */
+ private def skipExpr(expr: Expression): Boolean = {
+expr.isInstanceOf[LeafExpression] ||
// `LambdaVariable` is usually used as a loop variable, which can't be
evaluated ahead of the
// loop. So we can't evaluate sub-expressions containing
`LambdaVariable` at the beginning.
expr.find(_.isInstanceOf[LambdaVariable]).isDefined
+ }
+
+
+ // There are some special expressions that we should not recurse into all of
its children.
+ // 1. CodegenFallback: it's children will not be used to generate code
(call eval() instead)
+ // 2. If: common subexpressions will always be evaluated at the beginning,
but the true and
+ // false expressions in `If` may not get accessed, according to the
predicate
+ // expression. We should only recurse into the predicate expression.
+ // 3. CaseWhen: like `If`, the children of `CaseWhen` only get accessed in
a certain
+ //condition. We should only recurse into the first condition
expression as it
+ //will always get accessed.
+ // 4. Coalesce: it's also a conditional expression, we should only recurse
into the first
+ //children, because others may not get accessed.
+ private def childrenToRecurse(expr: Expression): Seq[Expression] = expr
match {
+case _: CodegenFallback => Nil
+case i: If => i.predicate :: Nil
+case c: CaseWhen => c.children.head :: Nil
+case c: Coalesce => c.children.head :: Nil
+case s: SortPrefix => s.child.child :: Nil
+case other => other.children
+ }
+
+ /**
+ * Adds the expression to this data structure recursively. Stops if a
matching expression
+ * is found. That is, if `expr` has already been added, its children are not
added.
+ */
+ def addExprTree(
+ expr: Expression,
+ exprMap: EquivalenceMap = this.equivalenceMap): Unit = {
+val skip = skipExpr(expr)
-// There are some special expressions that we should not recurse into all
of its children.
-// 1. CodegenFallback: it's children will not be used to generate code
(call eval() instead)
-// 2. If: common subexpressions will always be evaluated at the
beginning, but the true and
-// false expressions in `If` may not get accessed, according to
the predicate
-// expression. We should only recurse into the predicate
expression.
-// 3. CaseWhen: like `If`, the children of `CaseWhen` only get accessed
in a certain
-//condition. We
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r321996514 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala ## @@ -65,6 +65,40 @@ case class BoundReference(ordinal: Int, dataType: DataType, nullable: Boolean) } } +/** + * This bound reference points to a parameterized slot in an input tuple. It is used in + * common sub-expression elimination. When some common sub-expressions have same structural + * but different slots of input tuple, we replace `BoundReference` with this parameterized + * version. The slot position is parameterized and is given at runtime. + */ +case class ParameterizedBoundReference(parameter: String, dataType: DataType, nullable: Boolean) Review comment: Since this is only used for codegen, how about moving this to `org.apache.spark.sql.catalyst.expressions.codegen`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL]
Structurally equivalent subexpression elimination
URL: https://github.com/apache/spark/pull/25717#discussion_r321996570
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##
@@ -65,35 +89,102 @@ class EquivalentExpressions {
}
/**
- * Adds the expression to this data structure recursively. Stops if a
matching expression
- * is found. That is, if `expr` has already been added, its children are not
added.
+ * Adds each expression to structural expression data structure, grouping
them with existing
+ * structurally equivalent expressions. Non-recursive.
+ */
+ def addStructExpr(ctx: CodegenContext, expr: Expression): Unit = {
+if (expr.deterministic) {
+ val refs = expr.collect {
+case b: BoundReference => b
+ }
+
+ // For structural equivalent expressions, we need to pass in int type
ordinals into
+ // split functions. If the number of ordinals is more than JVM function
limit, we skip
+ // this expression.
+ // We calculate function parameter length by the number of ints plus
`INPUT_ROW` plus
+ // a int type result array index.
+ val parameterLength = CodeGenerator.calculateParamLength(refs.map(_ =>
Literal(0))) + 2
+ if (CodeGenerator.isValidParamLength(parameterLength)) {
+val parameterizedExpr = parameterizedBoundReferences(ctx, expr)
+
+val e: StructuralExpr = StructuralExpr(parameterizedExpr)
+val f = structEquivalenceMap.get(e)
+if (f.isDefined) {
+ addExpr(expr, f.get)
+} else {
+ val exprMap = mutable.HashMap.empty[Expr,
mutable.ArrayBuffer[Expression]]
+ addExpr(expr, exprMap)
+ structEquivalenceMap.put(e, exprMap)
+}
+ }
+}
+ }
+
+ /**
+ * Replaces bound references in given expression by parameterized bound
references.
*/
- def addExprTree(expr: Expression): Unit = {
-val skip = expr.isInstanceOf[LeafExpression] ||
+ private def parameterizedBoundReferences(ctx: CodegenContext, expr:
Expression): Expression = {
+expr.transformUp {
+ case b: BoundReference =>
+val param = ctx.freshName("boundInput")
Review comment:
nit: `boundInput` -> `ordinal`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination
maropu commented on a change in pull request #25717: [WIP][SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r321996545 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala ## @@ -65,6 +65,40 @@ case class BoundReference(ordinal: Int, dataType: DataType, nullable: Boolean) } } +/** + * This bound reference points to a parameterized slot in an input tuple. It is used in + * common sub-expression elimination. When some common sub-expressions have same structural + * but different slots of input tuple, we replace `BoundReference` with this parameterized + * version. The slot position is parameterized and is given at runtime. + */ +case class ParameterizedBoundReference(parameter: String, dataType: DataType, nullable: Boolean) Review comment: nit: `parameter` -> variableNameForOrdinal` or `paramNameForOrdinal`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529180155 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529180157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15297/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529180155 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file
AmplabJenkins commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529180157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15297/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file
SparkQA commented on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529179673 **[Test build #110302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110302/testReport)** for PR 25690 at commit [`b3613c0`](https://github.com/apache/spark/commit/b3613c077862ba58b15d49b4cd33d87f62384c2b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clien
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529177454 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
SparkQA commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178612 **[Test build #110301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110301/testReport)** for PR 25710 at commit [`3314954`](https://github.com/apache/spark/commit/3314954406203170fe2fff7ebf20a9e038bc689e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178610 **[Test build #110300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110300/testReport)** for PR 25720 at commit [`6051f0b`](https://github.com/apache/spark/commit/6051f0ba2c4e39effd55eaabe4d1799f4ecf84e0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clien
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178414 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clien
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15295/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178408 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178411 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15296/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178414 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178408 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-529178415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15295/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178411 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15296/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
maropu commented on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529178128 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15294/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177529 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15294/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177529 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177454 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110299/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177159 **[Test build #110299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110299/testReport)** for PR 25720 at commit [`87e3f64`](https://github.com/apache/spark/commit/87e3f64c074d46d6c9c203ee3093a6fb92f1f9d3). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177162 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
SparkQA removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177101 **[Test build #110299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110299/testReport)** for PR 25720 at commit [`87e3f64`](https://github.com/apache/spark/commit/87e3f64c074d46d6c9c203ee3093a6fb92f1f9d3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110299/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL'
AmplabJenkins removed a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capital form 'NULL' URL: https://github.com/apache/spark/pull/25720#issuecomment-529177162 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' as currently system cannot evaluate the last access time, and 'null' values will b
SparkQA commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' as currently system cannot evaluate the last access time, and 'null' values will be shown in its capital form 'NULL' for SQL client to make the display format similar to spark-shell. URL: https://github.com/apache/spark/pull/25720#issuecomment-529177101 **[Test build #110299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110299/testReport)** for PR 25720 at commit [`87e3f64`](https://github.com/apache/spark/commit/87e3f64c074d46d6c9c203ee3093a6fb92f1f9d3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sujith71955 opened a new pull request #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' as currently system cannot evaluate the last access time, and 'null' va
sujith71955 opened a new pull request #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' as currently system cannot evaluate the last access time, and 'null' values will be shown in its capital form 'NULL' for SQL client to make the display format similar to spark-shell. URL: https://github.com/apache/spark/pull/25720 **What changes were proposed in this pull request?** If there is no comment for spark scala shell shows "null" in small letters but all other places Hive beeline/Spark beeline/Spark SQL it is showing in CAPITAL "NULL". In this patch shown in its capital form 'NULL' for SQL client to make the display format similar to Hive beeline/Spark beeline/Spark SQL. Also corrected the Last Access time, the value shall display 'UNKNOWN' as currently system wont support the last access time evaluation. Issue 2 mentioned in JIRA Spark SQL "desc formatted tablename" is not showing the header # col_name,data_type,comment , seems to be the header has been removed knowingly as part of SPARK-20954. **Does this PR introduce any user-facing change?** No **How was this patch tested?** Locally and corrected a ut. Attaching the test report below  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sandeep-katta commented on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found
sandeep-katta commented on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found URL: https://github.com/apache/spark/pull/25399#issuecomment-529176790 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.
AmplabJenkins removed a comment on issue #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#issuecomment-529176351 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110298/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found
AmplabJenkins removed a comment on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found URL: https://github.com/apache/spark/pull/25399#issuecomment-529176358 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110297/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529176380 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110295/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file
AmplabJenkins removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529176384 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110292/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec
AmplabJenkins removed a comment on issue #25710: [SPARK-29008][SQL] Define an individual method for each common subexpression in HashAggregateExec URL: https://github.com/apache/spark/pull/25710#issuecomment-529176379 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file
SparkQA removed a comment on issue #25690: [WIP][SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529164457 **[Test build #110292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110292/testReport)** for PR 25690 at commit [`d5d0b7f`](https://github.com/apache/spark/commit/d5d0b7f83544e84a30fe3a3dfc0abb2aa1b5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found
SparkQA removed a comment on issue #25399: [SPARK-28670][SQL] create function should thrown Exception if the resource is not found URL: https://github.com/apache/spark/pull/25399#issuecomment-529172537 **[Test build #110297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110297/testReport)** for PR 25399 at commit [`eddf3fe`](https://github.com/apache/spark/commit/eddf3fea34a029ac255cc76be5fd2c1776eca796). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
