[GitHub] [spark] AmplabJenkins commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
AmplabJenkins commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823219548 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137673/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
SparkQA removed a comment on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823027651 **[Test build #137673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137673/testReport)** for PR 32247 at commit [`d636996`](https://github.com/apache/spark/commit/d6369965edcca517302b676a4c758fdd6f4be889). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
SparkQA commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823218259 **[Test build #137673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137673/testReport)** for PR 32247 at commit [`d636996`](https://github.com/apache/spark/commit/d6369965edcca517302b676a4c758fdd6f4be889). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan
AmplabJenkins removed a comment on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823216179 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137676/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
AmplabJenkins removed a comment on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823216181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42213/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation
AmplabJenkins removed a comment on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823216178 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137677/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
maropu commented on a change in pull request #32228:
URL: https://github.com/apache/spark/pull/32228#discussion_r616615387
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -772,9 +772,6 @@ object ColumnPruning extends Rule[LogicalPlan] {
f.copy(child = prunedChild(child, f.references))
case e @ Expand(_, _, child) if !child.outputSet.subsetOf(e.references) =>
e.copy(child = prunedChild(child, e.references))
-case s @ ScriptTransformation(_, _, _, child, _)
-if !child.outputSet.subsetOf(s.references) =>
- s.copy(child = prunedChild(child, s.references))
Review comment:
Ah, okay. I remembered that `ScriptTransformation` always has a
projected list of exprs in it's child now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation
AmplabJenkins commented on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823216178 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137677/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan
AmplabJenkins commented on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823216179 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137676/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
AmplabJenkins commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823216181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42213/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation
SparkQA removed a comment on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823033274 **[Test build #137677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137677/testReport)** for PR 32242 at commit [`c9d09be`](https://github.com/apache/spark/commit/c9d09be9ad22ddfe764195ce5e52a92dbda7df31). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation
SparkQA commented on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823212869 **[Test build #137677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137677/testReport)** for PR 32242 at commit [`c9d09be`](https://github.com/apache/spark/commit/c9d09be9ad22ddfe764195ce5e52a92dbda7df31). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823212702 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42213/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823209865 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42213/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan
SparkQA removed a comment on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823028148 **[Test build #137676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137676/testReport)** for PR 31451 at commit [`06eb9c7`](https://github.com/apache/spark/commit/06eb9c79a3fdd807ec08540deb2234939396325a). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan
SparkQA commented on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823196846 **[Test build #137676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137676/testReport)** for PR 31451 at commit [`06eb9c7`](https://github.com/apache/spark/commit/06eb9c79a3fdd807ec08540deb2234939396325a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime
AmplabJenkins removed a comment on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823189498 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137672/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
AmplabJenkins removed a comment on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823189278 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137670/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime
AmplabJenkins commented on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823189498 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137672/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
AmplabJenkins commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823189278 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137670/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime
SparkQA commented on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823188181 **[Test build #137672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137672/testReport)** for PR 32219 at commit [`42a95d0`](https://github.com/apache/spark/commit/42a95d0838fd7813afcbbfc9072a52de42069a52). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime
AmplabJenkins removed a comment on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823038173 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42200/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime
SparkQA removed a comment on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823005719 **[Test build #137672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137672/testReport)** for PR 32219 at commit [`42a95d0`](https://github.com/apache/spark/commit/42a95d0838fd7813afcbbfc9072a52de42069a52). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
SparkQA removed a comment on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823002404 **[Test build #137670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137670/testReport)** for PR 32247 at commit [`fe84f65`](https://github.com/apache/spark/commit/fe84f658f3904537749754e862b6f015d428027c). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules
SparkQA commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823187609 **[Test build #137670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137670/testReport)** for PR 32247 at commit [`fe84f65`](https://github.com/apache/spark/commit/fe84f658f3904537749754e862b6f015d428027c). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class ImmutableBitSet(val numBits: Int, val bitsToSet: Int*) extends BitSet(numBits) ` * ` case class CombinedTypeCoercionRule(rules: Seq[TypeCoercionRule]) extends TypeCoercionRule ` * `case class GroupingExprRef(` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13
SparkQA commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-823183791 **[Test build #137686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137686/testReport)** for PR 31569 at commit [`748bc9a`](https://github.com/apache/spark/commit/748bc9a61722a03738e0ce535dee956795b44883). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823183328 **[Test build #137685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137685/testReport)** for PR 32249 at commit [`b44429d`](https://github.com/apache/spark/commit/b44429dd4e1e5b70bb922cb279b489337af12886). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32202: [SPARK-28098][SQL]Supporting non-partitioned Hive tables with subdirectories
AmplabJenkins removed a comment on pull request #32202: URL: https://github.com/apache/spark/pull/32202#issuecomment-823182821 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42212/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32239: [SPARK-34639][SQL][3.1] RelationalGroupedDataset.alias should not create UnresolvedAlias
AmplabJenkins removed a comment on pull request #32239: URL: https://github.com/apache/spark/pull/32239#issuecomment-823182820 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42209/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32254: [SPARK-35143][ML] Removes the default number of partitions used for parallelize functions
AmplabJenkins commented on pull request #32254: URL: https://github.com/apache/spark/pull/32254#issuecomment-823183029 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32239: [SPARK-34639][SQL][3.1] RelationalGroupedDataset.alias should not create UnresolvedAlias
AmplabJenkins commented on pull request #32239: URL: https://github.com/apache/spark/pull/32239#issuecomment-823182820 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42209/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32202: [SPARK-28098][SQL]Supporting non-partitioned Hive tables with subdirectories
AmplabJenkins commented on pull request #32202: URL: https://github.com/apache/spark/pull/32202#issuecomment-823182821 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42212/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32202: [SPARK-28098][SQL]Supporting non-partitioned Hive tables with subdirectories
SparkQA commented on pull request #32202: URL: https://github.com/apache/spark/pull/32202#issuecomment-823181612 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42212/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] luhenry commented on pull request #32253: Accelerate fallback BLAS with dev.ludovic.netlib
luhenry commented on pull request #32253: URL: https://github.com/apache/spark/pull/32253#issuecomment-823181385 I'm adding support for JDK 8 with [v1.0.11](https://github.com/luhenry/netlib/tree/v1.0.11) (Benchmarks at https://github.com/luhenry/netlib/actions/runs/766849718) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32239: [SPARK-34639][SQL][3.1] RelationalGroupedDataset.alias should not create UnresolvedAlias
SparkQA commented on pull request #32239: URL: https://github.com/apache/spark/pull/32239#issuecomment-823178924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #32001: [SPARK-34902][SQL] Support cast between LongType & DayTimeIntervalType and IntegerType & YearMonthIntervalType
AngersZh commented on pull request #32001: URL: https://github.com/apache/spark/pull/32001#issuecomment-823172232 PG not support similar `cast`      -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31010: [SPARK-33976][SQL][DOCS] Add a SQL doc page for a TRANSFORM clause
AngersZh commented on pull request #31010: URL: https://github.com/apache/spark/pull/31010#issuecomment-823167454 @cloud-fan @maropu @srowen Thanks for your review, I will raise a new pr for master branch soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31010: [SPARK-33976][SQL][DOCS] Add a SQL doc page for a TRANSFORM clause
cloud-fan closed pull request #31010: URL: https://github.com/apache/spark/pull/31010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31010: [SPARK-33976][SQL][DOCS] Add a SQL doc page for a TRANSFORM clause
cloud-fan commented on pull request #31010: URL: https://github.com/apache/spark/pull/31010#issuecomment-823166555 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ChenDou2021 opened a new pull request #32254: [SPARK-35143][ML] Removes the default number of partitions used for parallelize functions
ChenDou2021 opened a new pull request #32254: URL: https://github.com/apache/spark/pull/32254 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk closed pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
MaxGekk closed pull request #32250: URL: https://github.com/apache/spark/pull/32250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
MaxGekk commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823159245 +1, LGTM. Merging to master. Thank you, @AngersZh. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823158491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42211/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32202: [SPARK-28098][SQL]Supporting non-partitioned Hive tables with subdirectories
SparkQA commented on pull request #32202: URL: https://github.com/apache/spark/pull/32202#issuecomment-823158600 **[Test build #137684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137684/testReport)** for PR 32202 at commit [`7cc9c95`](https://github.com/apache/spark/commit/7cc9c959345ce28c3ae380a46388f049f9971ac7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823158464 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42211/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823158491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42211/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
maropu commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823157605 Looks fine if the failure fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
AngersZh commented on a change in pull request #32228:
URL: https://github.com/apache/spark/pull/32228#discussion_r616546346
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -772,9 +772,6 @@ object ColumnPruning extends Rule[LogicalPlan] {
f.copy(child = prunedChild(child, f.references))
case e @ Expand(_, _, child) if !child.outputSet.subsetOf(e.references) =>
e.copy(child = prunedChild(child, e.references))
-case s @ ScriptTransformation(_, _, _, child, _)
-if !child.outputSet.subsetOf(s.references) =>
- s.copy(child = prunedChild(child, s.references))
Review comment:
> Why don't we need the pruning?
it pruning input to child, child's output always transform's input, so the
column pruning for script transform is not useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823156676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42210/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823156676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42210/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823156542 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42210/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823155907 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137683/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823151934 **[Test build #137683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137683/testReport)** for PR 31638 at commit [`1c26b2a`](https://github.com/apache/spark/commit/1c26b2a93c412559897d63229727315ff493fc98). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823155907 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137683/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823155869 **[Test build #137683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137683/testReport)** for PR 31638 at commit [`1c26b2a`](https://github.com/apache/spark/commit/1c26b2a93c412559897d63229727315ff493fc98). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
maropu commented on a change in pull request #32228:
URL: https://github.com/apache/spark/pull/32228#discussion_r616541698
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -772,9 +772,6 @@ object ColumnPruning extends Rule[LogicalPlan] {
f.copy(child = prunedChild(child, f.references))
case e @ Expand(_, _, child) if !child.outputSet.subsetOf(e.references) =>
e.copy(child = prunedChild(child, e.references))
-case s @ ScriptTransformation(_, _, _, child, _)
-if !child.outputSet.subsetOf(s.references) =>
- s.copy(child = prunedChild(child, s.references))
Review comment:
Why don't we need the pruning?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823149624 **[Test build #137682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137682/testReport)** for PR 31638 at commit [`7c40405`](https://github.com/apache/spark/commit/7c404057827b679bbd47f5a01e5994c8ccabf471). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins removed a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823153961 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137682/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
AmplabJenkins commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823153961 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137682/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823153925 **[Test build #137682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137682/testReport)** for PR 31638 at commit [`7c40405`](https://github.com/apache/spark/commit/7c404057827b679bbd47f5a01e5994c8ccabf471). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
AmplabJenkins removed a comment on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823153165 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137678/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu removed a comment on pull request #32252: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
AngersZh removed a comment on pull request #32252: URL: https://github.com/apache/spark/pull/32252#issuecomment-823140972 test this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
AmplabJenkins commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823153165 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137678/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
SparkQA removed a comment on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823067877 **[Test build #137678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137678/testReport)** for PR 32249 at commit [`b8ef281`](https://github.com/apache/spark/commit/b8ef281da64ba42bb37992603675ecf9c7d6dff7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking
SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823152508 **[Test build #137678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137678/testReport)** for PR 32249 at commit [`b8ef281`](https://github.com/apache/spark/commit/b8ef281da64ba42bb37992603675ecf9c7d6dff7). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823151934 **[Test build #137683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137683/testReport)** for PR 31638 at commit [`1c26b2a`](https://github.com/apache/spark/commit/1c26b2a93c412559897d63229727315ff493fc98). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #32202: [SPARK-28098][SQL]Supporting non-partitioned Hive tables with subdirectories
attilapiros commented on pull request #32202: URL: https://github.com/apache/spark/pull/32202#issuecomment-823151771 cc @peter-toth -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
xuanyuanking commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823151538 @HeartSaVioR Agree. Updated has done. Let's fix the regression in this PR first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31638: [SPARK-34526][SS] Ignore the error when checking the path in FileStreamSink.hasMetadata
SparkQA commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-823149624 **[Test build #137682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137682/testReport)** for PR 31638 at commit [`7c40405`](https://github.com/apache/spark/commit/7c404057827b679bbd47f5a01e5994c8ccabf471). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI
AmplabJenkins removed a comment on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823038348 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42197/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite
AmplabJenkins removed a comment on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823148566 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137674/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
AmplabJenkins removed a comment on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823148559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32239: [SPARK-34639][SQL][3.1] RelationalGroupedDataset.alias should not create UnresolvedAlias
SparkQA commented on pull request #32239: URL: https://github.com/apache/spark/pull/32239#issuecomment-823149091 **[Test build #137681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137681/testReport)** for PR 32239 at commit [`63cac4d`](https://github.com/apache/spark/commit/63cac4d0c13214cec2b9bed38ea848651dbe387b). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
AmplabJenkins removed a comment on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823148562 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42208/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI
AmplabJenkins removed a comment on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823148563 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137669/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
AmplabJenkins commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823148562 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42208/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI
AmplabJenkins commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823148563 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137669/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32252: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
AmplabJenkins commented on pull request #32252: URL: https://github.com/apache/spark/pull/32252#issuecomment-823148728 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32253: Accelerate fallback BLAS with dev.ludovic.netlib
AmplabJenkins commented on pull request #32253: URL: https://github.com/apache/spark/pull/32253#issuecomment-823148693 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
AmplabJenkins commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823148560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite
AmplabJenkins commented on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823148566 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137674/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Peng-Lei commented on pull request #32252: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
Peng-Lei commented on pull request #32252: URL: https://github.com/apache/spark/pull/32252#issuecomment-823147189 > pls change UT to `[SPARK-35094][SQL] xxx` @AngersZh done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Peng-Lei commented on pull request #32252: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
Peng-Lei commented on pull request #32252: URL: https://github.com/apache/spark/pull/32252#issuecomment-823146691 @MaxGekk Hi, could you please help to review on this ? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
SparkQA commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823145628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite
SparkQA removed a comment on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823027790 **[Test build #137674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137674/testReport)** for PR 32146 at commit [`ccd4e0a`](https://github.com/apache/spark/commit/ccd4e0a060fd9d91f74f53ccdb0423299be0d12c). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] luhenry opened a new pull request #32253: Accelerate fallback BLAS with dev.ludovic.netlib
luhenry opened a new pull request #32253: URL: https://github.com/apache/spark/pull/32253 ### What changes were proposed in this pull request? Following https://github.com/apache/spark/pull/30810, I've continued looking for ways to accelerate the usage of BLAS in Spark. With this PR, I integrate work done in the [`dev.ludovic.netlib`](https://github.com/luhenry/netlib/) Maven package. The `dev.ludovic.netlib` library wraps the original `com.github.fommil.netlib` library and focus on accelerating the linear algebra routines in use in Spark. When running the `org.apache.spark.ml.linalg.BLASBenchmark` benchmarking suite, I get the results at [1] on an Intel machine. Moreover, this library is thoroughly tested to return the exact same results as the reference implementation. Under the hood, it reimplements the necessary algorithms in pure autovectorization-friendly Java 8, as well as takes advantage of the Vector API and Foreign Linker API introduced in JDK 16 when available. ### Why are the changes needed? Accelerates linear algebra operations when the pure-java fallback method is in use. Transparently falls back to native implementation (OpenBLAS, MKL) when available. ### Does this PR introduce _any_ user-facing change? No, all changes are transparent to the user. ### How was this patch tested? The `dev.ludovic.netlib` library has its own test suite [2]. It has also been validated by running the Spark test suite and benchmarking suite. [1] https://gist.github.com/luhenry/691d1f497595b3404a6180b6c57336e0 [2] https://github.com/luhenry/netlib/tree/master/blas/src/test/java/dev/ludovic/netlib/blas -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite
SparkQA commented on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823142674 **[Test build #137674 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137674/testReport)** for PR 32146 at commit [`ccd4e0a`](https://github.com/apache/spark/commit/ccd4e0a060fd9d91f74f53ccdb0423299be0d12c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class ParquetEncryptionSuite extends QueryTest with TestHiveSingleton ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
SparkQA commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823142553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #32252: fix spark-35094
AngersZh commented on pull request #32252: URL: https://github.com/apache/spark/pull/32252#issuecomment-823140902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final
LuciferYang commented on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823137116 @dongjoon-hyun @HyukjinKwon @srowen All test passed now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #31989: [SPARK-34891][SS] Introduce state store manager for session window in streaming query
xuanyuanking commented on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-823135370 @viirya Sorry for the late. Just back from the work on open-sourcing RocksDBStateStore. Since the design has been published. I can back to this now. I'm +1 for the `StreamingSessionWindowStateManager` abstraction. Only have some concern about the StreamingSessionWindowStateManagerImplV1. It would be great if we can have a better state store API to reduce the complexity of the implementation. I'll start working on it in these two weeks (in parallel with open-sourcing the RocksDB state store). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
SparkQA removed a comment on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823109907 **[Test build #137679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137679/testReport)** for PR 32250 at commit [`b1a81bd`](https://github.com/apache/spark/commit/b1a81bd84452a2f52cd8546e9e3d852d9b4f77db). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite
SparkQA commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823134873 **[Test build #137679 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137679/testReport)** for PR 32250 at commit [`b1a81bd`](https://github.com/apache/spark/commit/b1a81bd84452a2f52cd8546e9e3d852d9b4f77db). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Peng-Lei opened a new pull request #32252: fix spark-35094
Peng-Lei opened a new pull request #32252: URL: https://github.com/apache/spark/pull/32252 ### What changes were proposed in this pull request? In https://issues.apache.org/jira/browse/SPARK-35094, When use spark from_json(JsonToStruct) function in permissive mode to handle the case of contains incorrect nested json fields. It will return wrong value. ### Why are the changes needed? It is necessary to ensure that the data is correct. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? unit test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI
SparkQA removed a comment on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-822997550 **[Test build #137669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137669/testReport)** for PR 31974 at commit [`eadb6da`](https://github.com/apache/spark/commit/eadb6dab935b26ffcae4e85f027ae324b98536ba). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework
SparkQA removed a comment on pull request #32072: URL: https://github.com/apache/spark/pull/32072#issuecomment-822956888 **[Test build #137667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137667/testReport)** for PR 32072 at commit [`f665a0d`](https://github.com/apache/spark/commit/f665a0d52953ea7927d0d5dee7f4742255173691). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Peng-Lei closed pull request #32251: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
Peng-Lei closed pull request #32251: URL: https://github.com/apache/spark/pull/32251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Peng-Lei opened a new pull request #32251: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode
Peng-Lei opened a new pull request #32251: URL: https://github.com/apache/spark/pull/32251 ### Why are the changes needed? In https://issues.apache.org/jira/browse/SPARK-35094, When use spark from_json(JsonToStruct) function in permissive mode to handle the case of contains incorrect nested json fields. It will return wrong value. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? unit test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI
SparkQA commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823125287 **[Test build #137669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137669/testReport)** for PR 31974 at commit [`eadb6da`](https://github.com/apache/spark/commit/eadb6dab935b26ffcae4e85f027ae324b98536ba). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class SparkListenerMiscellaneousProcessAdded(time: Long, processId: String,` * ` case class MiscellaneousProcessAdded(` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework
cloud-fan closed pull request #32072: URL: https://github.com/apache/spark/pull/32072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
