[GitHub] [spark] MaxGekk commented on issue #27807: [WIP][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time
MaxGekk commented on issue #27807: [WIP][SQL] Convert Catalyst's DATE/TIMESTAMP to Java Date/Timestamp via local date-time URL: https://github.com/apache/spark/pull/27807#issuecomment-595648475 @cloud-fan The rebasing cause a few issues. For example, we cannot use TimestampFormatter/DateFormatter to format java.sql.Timestamp/Date anymore: https://github.com/apache/spark/blob/1fac06c4307c8c7a5a48a50952d48ee5b9ebccb2/sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala#L74 To format rebased Java Timestamp/Date instances before 1582, need to use Julian based formatter - SimpleDateFormat. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
AmplabJenkins removed a comment on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828#issuecomment-595648035 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24184/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
AmplabJenkins commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828#issuecomment-595648035 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24184/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
AmplabJenkins commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828#issuecomment-595648029 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
AmplabJenkins removed a comment on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828#issuecomment-595648029 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
maropu commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-595648132 Thanks for the reviews, @HyukjinKwon and @viirya! Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu closed pull request #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID
maropu closed pull request #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
SparkQA commented on issue #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828#issuecomment-595647574 **[Test build #119449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119449/testReport)** for PR 27828 at commit [`19b13ce`](https://github.com/apache/spark/commit/19b13cefa65f3687e10ceff9ffbe556ede94f9f8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin opened a new pull request #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
LantaoJin opened a new pull request #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange URL: https://github.com/apache/spark/pull/27828 ### What changes were proposed in this pull request? Fix the IllegalArgumentException in broadcast exchange when numRows over 341 million but less than 512 million. Since the maximum number of keys that `BytesToBytesMap` supports is 1 << 29, and only 70% of the slots can be used before growing in `HashedRelation`, So here the limitation should not be greater equal than 341 million (1 << 29 / 1.5(357913941)) instead of 512 million. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Manually test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
maropu commented on a change in pull request #27827: [SPARK-31066] Disable
useless and uncleaned hive SessionState initialization parts
URL: https://github.com/apache/spark/pull/27827#discussion_r388754145
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##
@@ -1232,4 +1204,42 @@ private[hive] object HiveClientImpl {
StatsSetupConst.RAW_DATA_SIZE,
StatsSetupConst.TOTAL_SIZE
)
+
+ def newHiveConf(
+ sparkConf: SparkConf,
+ hadoopConf: JIterable[JMap.Entry[String, String]],
+ extraConfig: Map[String, String],
+ classLoader: Option[ClassLoader] = None): HiveConf = {
+val hiveConf = new HiveConf(classOf[SessionState])
+// HiveConf is a Hadoop Configuration, which has a field of classLoader and
+// the initial value will be the current thread's context class loader.
+// We call hiveConf.setClassLoader(initClassLoader) at here to ensure it
use the classloader
+// we want.
+classLoader.foreach(hiveConf.setClassLoader)
+// 1: Take all from the hadoopConf to this hiveConf.
+// This hadoopConf contains user settings in Hadoop's core-site.xml file
+// and Hive's hive-site.xml file. Note, we load hive-site.xml file
manually in
+// SharedState and put settings in this hadoopConf instead of relying on
HiveConf
+// to load user settings. Otherwise, HiveConf's initialize method will
override
+// settings in the hadoopConf. This issue only shows up when
spark.sql.hive.metastore.jars
+// is not set to builtin. When spark.sql.hive.metastore.jars is builtin,
the classpath
+// has hive-site.xml. So, HiveConf will use that to override its default
values.
+// 2: we set all spark confs to this hiveConf.
+// 3: we set all entries in config to this hiveConf.
+val confMap = (hadoopConf.iterator().asScala.map(kv => kv.getKey ->
kv.getValue) ++
+ sparkConf.getAll.toMap ++ extraConfig).toMap
+confMap.foreach { case (k, v) => hiveConf.set(k, v) }
+SQLConf.get.redactOptions(confMap).foreach { case (k, v) =>
+ logDebug(s"Applying Hadoop/Hive/Spark and extra properties to Hive
Conf:$k=$v")
+}
+// Disable CBO because we removed the Calcite dependency.
+hiveConf.setBoolean("hive.cbo.enable", false)
+// If this is true, SessionState.start will create a file to log hive job
which will not be
+// deleted on exit and is useless for spark
+hiveConf.setBoolean("hive.session.history.enabled", false)
+// If this is tez engine, SessionState.start might bring extra logic to
initialize tez stuff,
+// which is useless for spark.
+hiveConf.set("hive.execution.engine", "spark")
Review comment:
The overwrite is better than just a warning log?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade
gengliangwang commented on issue #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade URL: https://github.com/apache/spark/pull/27370#issuecomment-595644137 I manually test this PR change again. I think it is almost ready and will check more closely again before get it merged. @srowen would you like to take a look at this as well? Also, it seems still a bit risky to put it into 3.0 so I am thinking about making this in master only for now. WDYT? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27822: [SPARK-31061][SQL] Provide ability to alter the provider of a table
cloud-fan commented on issue #27822: [SPARK-31061][SQL] Provide ability to alter the provider of a table URL: https://github.com/apache/spark/pull/27822#issuecomment-595643358 thanks, merging to master/3.0! (I've fixed the typo in the test while merging) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27822: [SPARK-31061][SQL] Provide ability to alter the provider of a table
cloud-fan closed pull request #27822: [SPARK-31061][SQL] Provide ability to alter the provider of a table URL: https://github.com/apache/spark/pull/27822 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade
gengliangwang commented on issue #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade URL: https://github.com/apache/spark/pull/27370#issuecomment-595642935 @clarkead I noticed that this PR rename the jquery.dataTables.1.10.18.min.js(css) to jquery.dataTables.min.js(css). How about keeping the version in the file name? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
AmplabJenkins removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595642454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119439/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
maropu commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595643254 also cc: @wangyum This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
AmplabJenkins removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595642448 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
SparkQA removed a comment on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595573088 **[Test build #119439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119439/testReport)** for PR 27432 at commit [`ce13f8e`](https://github.com/apache/spark/commit/ce13f8eedd93e1ead664fd427e2ce10a9d350f11). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
AmplabJenkins commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595642454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119439/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
AmplabJenkins commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595642448 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax
SparkQA commented on issue #27432: [SPARK-28325][SQL]Support ANSI SQL: SIMILAR TO ... ESCAPE syntax URL: https://github.com/apache/spark/pull/27432#issuecomment-595642089 **[Test build #119439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119439/testReport)** for PR 27432 at commit [`ce13f8e`](https://github.com/apache/spark/commit/ce13f8eedd93e1ead664fd427e2ce10a9d350f11). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade
gengliangwang commented on a change in pull request #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade URL: https://github.com/apache/spark/pull/27370#discussion_r388751502 ## File path: licenses/LICENSE-bootstrap.txt ## @@ -1,13 +1,22 @@ -Copyright 2013 Twitter, Inc. +The MIT License (MIT) Review comment: We should also update here: https://github.com/apache/spark/blob/master/LICENSE-binary#L419 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #27813: [SPARK-31054][REPL] Turn on deprecation in Scala REPL/SparkShell by default
Ngone51 commented on issue #27813: [SPARK-31054][REPL] Turn on deprecation in Scala REPL/SparkShell by default URL: https://github.com/apache/spark/pull/27813#issuecomment-595634208 Thank you guys for all your inputs here. It make sense to me about intrusive warnings from third-party and optionally control. Since I cannot find better way to improve and address your concern, I'm going to close this. Thanks, anyway!! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 closed pull request #27813: [SPARK-31054][REPL] Turn on deprecation in Scala REPL/SparkShell by default
Ngone51 closed pull request #27813: [SPARK-31054][REPL] Turn on deprecation in Scala REPL/SparkShell by default URL: https://github.com/apache/spark/pull/27813 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] BdLearnerr commented on issue #26029: [SPARK-29336][SQL] Fix the implementation of QuantileSummaries.merge (guarantee that the relativeError will be respected)
BdLearnerr commented on issue #26029: [SPARK-29336][SQL] Fix the implementation of QuantileSummaries.merge (guarantee that the relativeError will be respected) URL: https://github.com/apache/spark/pull/26029#issuecomment-595632758 anyone tell me , if this is incorporated in 2.4.4 version ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
AmplabJenkins removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595629196 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
AmplabJenkins removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595629198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119437/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
AmplabJenkins commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595629198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119437/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
AmplabJenkins commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595629196 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
AmplabJenkins commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595628033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24183/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
SparkQA removed a comment on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595552853 **[Test build #119437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119437/testReport)** for PR 27657 at commit [`e255906`](https://github.com/apache/spark/commit/e2559060d8fd99078d9b25472c45072867531fe8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
AmplabJenkins removed a comment on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595628023 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
AmplabJenkins commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595628023 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
AmplabJenkins removed a comment on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595628033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24183/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get
SparkQA commented on issue #27657: [SPARK-30899][SQL] CreateArray/CreateMap's data type should not depend on SQLConf.get URL: https://github.com/apache/spark/pull/27657#issuecomment-595627913 **[Test build #119437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119437/testReport)** for PR 27657 at commit [`e255906`](https://github.com/apache/spark/commit/e2559060d8fd99078d9b25472c45072867531fe8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
SparkQA commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595627651 **[Test build #119448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119448/testReport)** for PR 27827 at commit [`36c697d`](https://github.com/apache/spark/commit/36c697d394fc11893c25fc009739677fa9b1fa28). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
SparkQA removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595581162 **[Test build #119440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119440/testReport)** for PR 27810 at commit [`0744be7`](https://github.com/apache/spark/commit/0744be739dbf1cfbd884d3ddbd63894e468a4231). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595626868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119440/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595626862 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
yaooqinn commented on issue #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827#issuecomment-595626966 cc @cloud-fan @maropu @dongjoon-hyun, thanks for reviewing this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595626740 **[Test build #119440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119440/testReport)** for PR 27810 at commit [`0744be7`](https://github.com/apache/spark/commit/0744be739dbf1cfbd884d3ddbd63894e468a4231). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595626862 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595626868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119440/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn opened a new pull request #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts
yaooqinn opened a new pull request #27827: [SPARK-31066] Disable useless and uncleaned hive SessionState initialization parts URL: https://github.com/apache/spark/pull/27827 ### What changes were proposed in this pull request? As a common usage and according to the spark doc, users may often just copy their `hive-site.xml` to Spark directly from hive projects. Sometimes, the config file is not that clean for spark and may cause some side effects. for example, `hive.session.history.enabled` will create a log for the hive jobs but useless for spark and also it will not be deleted on JVM exit. this pr 1)disable `hive.session.history.enabled` explicitly to disable creating `hive_job_log` file, e.g. ``` Hive history file=/var/folders/01/h81cs4sn3dq2dd_k4j6fhrmcgn/T//kentyao/hive_job_log_79c63b29-95a4-4935-a9eb-2d89844dfe4f_493861201.txt ``` 2) set `hive.execution.engine` to `spark` explicitly in case the config is `tez` and casue uneccesary problem like this: ``` Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tez/dag/api/SessionNotRunning at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:529) ``` ### Why are the changes needed? reduce overhead of internal complexity and users' hive cognitive load for running spark ### Does this PR introduce any user-facing change? yes, `hive_job_log` file will not be created even enabled, and will not try to initialize tez kinds of stuff ### How was this patch tested? add ut and verify manually This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119447/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
SparkQA removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623593 **[Test build #119447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119447/testReport)** for PR 27810 at commit [`d257d26`](https://github.com/apache/spark/commit/d257d262a02ecd2475fd105e0ff05ebc5113d8df). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595619717 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins removed a comment on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623631 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595619722 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119426/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623631 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623615 **[Test build #119447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119447/testReport)** for PR 27810 at commit [`d257d26`](https://github.com/apache/spark/commit/d257d262a02ecd2475fd105e0ff05ebc5113d8df). * This patch **fails due to an unknown error code, 127**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
SparkQA commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623593 **[Test build #119447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119447/testReport)** for PR 27810 at commit [`d257d26`](https://github.com/apache/spark/commit/d257d262a02ecd2475fd105e0ff05ebc5113d8df). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins
AmplabJenkins commented on issue #27810: DO NOT MERGE: Testing KafkaDelegationTokenSuite with testRetry in Jenkins URL: https://github.com/apache/spark/pull/27810#issuecomment-595623637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119447/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595619722 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119426/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595619717 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
SparkQA removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595502171 **[Test build #119426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119426/testReport)** for PR 27817 at commit [`0a96333`](https://github.com/apache/spark/commit/0a963330d814b5660a2828d40c4e3244baece6b7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
SparkQA commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595619017 **[Test build #119426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119426/testReport)** for PR 27817 at commit [`0a96333`](https://github.com/apache/spark/commit/0a963330d814b5660a2828d40c4e3244baece6b7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595617814 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595617819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119433/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595617819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119433/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
AmplabJenkins commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595617814 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
cloud-fan commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595617798 sorry to be late, thanks for fixing! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
SparkQA removed a comment on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595515316 **[Test build #119433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119433/testReport)** for PR 27817 at commit [`83c2656`](https://github.com/apache/spark/commit/83c2656f569a2d7e8e686cdf7ed5ff5c0e5d47aa). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter`
SparkQA commented on issue #27817: [SPARK-31060] [SQL] [test-hive1.2] Handle column names containing `dots` in data source `Filter` URL: https://github.com/apache/spark/pull/27817#issuecomment-595617162 **[Test build #119433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119433/testReport)** for PR 27817 at commit [`83c2656`](https://github.com/apache/spark/commit/83c2656f569a2d7e8e686cdf7ed5ff5c0e5d47aa). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels
huaxingao commented on issue #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels URL: https://github.com/apache/spark/pull/27679#issuecomment-595611899 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
AmplabJenkins removed a comment on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595609084 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
AmplabJenkins removed a comment on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595609090 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24182/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
AmplabJenkins commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595609090 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24182/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
AmplabJenkins commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595609084 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
SparkQA commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595608777 **[Test build #119446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119446/testReport)** for PR 27825 at commit [`5dc0a7d`](https://github.com/apache/spark/commit/5dc0a7d4d6faa1627df7409872ec78c1b5840e15). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
HeartSaVioR commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607736 Thanks for the quick review and merge! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
dongjoon-hyun commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595607738 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
dongjoon-hyun commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595607708 Thanks, @HeartSaVioR . The PR is merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
dongjoon-hyun closed pull request #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
AmplabJenkins removed a comment on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607307 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
AmplabJenkins removed a comment on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24181/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
AmplabJenkins commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607307 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
AmplabJenkins commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24181/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
SparkQA commented on issue #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826#issuecomment-595607066 **[Test build #119445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119445/testReport)** for PR 27826 at commit [`6c367ab`](https://github.com/apache/spark/commit/6c367ab1add5d4bd0a77226835fe33ec1a5d49f7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595606219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119435/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels
zhengruifeng closed pull request #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels URL: https://github.com/apache/spark/pull/27679 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595606213 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
SparkQA removed a comment on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595521799 **[Test build #119435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119435/testReport)** for PR 27780 at commit [`4cc2ff6`](https://github.com/apache/spark/commit/4cc2ff6cc23158895f0991b9042e01b7db46b84f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels
zhengruifeng commented on issue #27679: [SPARK-30776][ML] Support FValueSelector for continuous features and continuous labels URL: https://github.com/apache/spark/pull/27679#issuecomment-595606280 Merged to master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595606213 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
AmplabJenkins commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595606219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119435/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach
HeartSaVioR commented on issue #27825: [SPARK-31014][CORE][3.0] InMemoryStore: remove key from parentToChildrenMap when removing key from CountingRemoveIfForEach URL: https://github.com/apache/spark/pull/27825#issuecomment-595606047 #27826 to fix build failure. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
SparkQA commented on issue #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots URL: https://github.com/apache/spark/pull/27780#issuecomment-595606027 **[Test build #119435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119435/testReport)** for PR 27780 at commit [`4cc2ff6`](https://github.com/apache/spark/commit/4cc2ff6cc23158895f0991b9042e01b7db46b84f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public final class SparkFilterApi ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27798: [SPARK-31045][SQL] Add config for AQE logging level
HeartSaVioR commented on issue #27798: [SPARK-31045][SQL] Add config for AQE logging level URL: https://github.com/apache/spark/pull/27798#issuecomment-595605960 I just fixed it via #27826 coincidentally - please take a look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
AmplabJenkins removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595605589 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
dongjoon-hyun closed pull request #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
AmplabJenkins removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595605592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119427/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR opened a new pull request #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0
HeartSaVioR opened a new pull request #27826: [SPARK-31045][SQL][FOLLOWUP][3.0] Fix build due to divergence between master and 3.0 URL: https://github.com/apache/spark/pull/27826 ### What changes were proposed in this pull request? This patch fixes the build failure in `branch-3.0` due to cherry-picking SPARK-31045 to branch-3.0, as `.version()` is not available in `branch-3.0` yet. ### Why are the changes needed? The build is failing in `branch-3.0`. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Jenkins build will verify. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
AmplabJenkins commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595605592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119427/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
AmplabJenkins commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595605589 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
SparkQA removed a comment on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595502184 **[Test build #119427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119427/testReport)** for PR 27816 at commit [`fe64572`](https://github.com/apache/spark/commit/fe64572354d6c9aec392f46ed236e8184572e147). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite
SparkQA commented on issue #27816: [SPARK-23817][FollowUp][Test] Add OrcV2QuerySuite URL: https://github.com/apache/spark/pull/27816#issuecomment-595605037 **[Test build #119427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119427/testReport)** for PR 27816 at commit [`fe64572`](https://github.com/apache/spark/commit/fe64572354d6c9aec392f46ed236e8184572e147). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class OrcQuerySuite extends OrcQueryTest with SharedSparkSession ` * `class OrcV2QuerySuite extends OrcQuerySuite ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27824: [SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support
AmplabJenkins removed a comment on issue #27824: [SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support URL: https://github.com/apache/spark/pull/27824#issuecomment-595604209 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27824: [SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support
AmplabJenkins removed a comment on issue #27824: [SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support URL: https://github.com/apache/spark/pull/27824#issuecomment-595604212 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24180/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dbtsai commented on a change in pull request #27824: [SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support
dbtsai commented on a change in pull request #27824: [SPARK-31064] [SQL] New
Parquet Predicate Filter APIs with multi-part Identifier Support
URL: https://github.com/apache/spark/pull/27824#discussion_r388716782
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##
@@ -126,264 +126,270 @@ class ParquetFilters(
Binary.fromConstantByteArray(fixedLengthBytes, 0, numBytes)
}
- private val makeEq: PartialFunction[ParquetSchemaType, (String, Any) =>
FilterPredicate] = {
+ private val makeEq
+ : PartialFunction[ParquetSchemaType, (Array[String], Any) =>
FilterPredicate] = {
Review comment:
I decide not to do type alias because it's easier to see the actual type
definition of function.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
