[GitHub] [spark] viirya commented on a change in pull request #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
viirya commented on a change in pull request #27499: [SPARK-30590][SQL] Untyped
select API cannot take typed column expression
URL: https://github.com/apache/spark/pull/27499#discussion_r382897674
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##
@@ -1430,6 +1430,11 @@ class Dataset[T] private[sql](
*/
@scala.annotation.varargs
def select(cols: Column*): DataFrame = withPlan {
+cols.find(_.isInstanceOf[TypedColumn[_, _]]).foreach { typedCol =>
+ throw new AnalysisException(s"$typedCol is a typed column that " +
+"cannot be passed in untyped `select` API. If you are going to select
" +
+"multiple typed columns, you can use `Dataset.selectUntyped` API.")
+}
Review comment:
We already shipped `count` as `TypedColumn` for a while, now to change it to
`Column` is a breaking change.
In order to not break current usage, I think the fix is to allow feasible
typed columns (e.g. `count`) in the untyped `select`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
viirya commented on a change in pull request #27499: [SPARK-30590][SQL] Untyped
select API cannot take typed column expression
URL: https://github.com/apache/spark/pull/27499#discussion_r382898306
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##
@@ -1430,7 +1430,23 @@ class Dataset[T] private[sql](
*/
@scala.annotation.varargs
def select(cols: Column*): DataFrame = withPlan {
-Project(cols.map(_.named), logicalPlan)
+val untypedCols = cols.map {
+ case typedCol: TypedColumn[_, _] =>
Review comment:
In order to not make a breaking change, we need to accept simple typed
columns like `count`. @cloud-fan @HyukjinKwon
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934696 **[Test build #118813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118813/testReport)** for PR 27499 at commit [`096ce42`](https://github.com/apache/spark/commit/096ce420f8e5d04dac1b88e46c53820ace857507). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23563/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934785 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934785 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23563/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
SparkQA commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938369 **[Test build #118814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118814/testReport)** for PR 27672 at commit [`a05af57`](https://github.com/apache/spark/commit/a05af57179adc2b0812841e71d04b9c818665d84). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938454 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938457 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23564/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938454 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938457 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23564/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer opened a new pull request #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
beliefer opened a new pull request #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674 ### What changes were proposed in this pull request? 1.Add version information to the configuration of `Network`. 2.Update the docs of `Network`. I sorted out some information show below. Item name | Since version | JIRA ID | Commit ID | Note -- | -- | -- | -- | -- spark.network.crypto.saslFallback | 2.2.0 | SPARK-19139 | 8f3f73abc1fe62496722476460c174af0250e3fe#diff-0ac65da2bc6b083fb861fe410c7688c2 | spark.network.crypto.enabled | 2.2.0 | SPARK-19139 | 8f3f73abc1fe62496722476460c174af0250e3fe#diff-6bdad48cfc34314e89599655442ff210 | spark.network.remoteReadNioBufferConversion | 2.4.0 | SPARK-24307 | 2c82745686f4456c4d5c84040a431dcb5b6cb60b#diff-2b643ea78c1add0381754b1f47eec132 | spark.network.timeout | 1.3.0 | SPARK-4688 | d3f07fd23cc26a70f44c52e24445974d4885d58a#diff-1df6b5af3d8f9f16255ff8c7a06f402f | spark.network.timeoutInterval | 1.3.2 | SPARK-5529 | ec196ab1c7569d7ab0a50c9d7338c2835f2c84d5#diff-47779b72f095f7e7f926898fa1a425ee | spark.rpc.askTimeout | 1.4.0 | SPARK-6490 | 8136810dfad12008ac300116df7bc8448740f1ae#diff-529fc5c06b9731c1fbda6f3db60b16aa | spark.rpc.connect.threads | 1.6.0 | SPARK-6028 | 084e4e126211d74a79e8dbd2d0e604dd3c650822#diff-0c89b4a60c30a7cd2224bb64d93da942 | spark.rpc.io.numConnectionsPerPeer | 1.6.0 | SPARK-10745 | 34a77679877bc40b58a10ec539a8da00fed7db39#diff-0c89b4a60c30a7cd2224bb64d93da942 | spark.rpc.io.threads | 1.6.0 | SPARK-6028 | 084e4e126211d74a79e8dbd2d0e604dd3c650822#diff-0c89b4a60c30a7cd2224bb64d93da942 | spark.rpc.lookupTimeout | 1.4.0 | SPARK-6490 | 8136810dfad12008ac300116df7bc8448740f1ae#diff-529fc5c06b9731c1fbda6f3db60b16aa | spark.rpc.message.maxSize | 2.0.0 | SPARK-7997 | bc1babd63da4ee56e6d371eb24805a5d714e8295#diff-529fc5c06b9731c1fbda6f3db60b16aa | spark.rpc.netty.dispatcher.numThreads | 1.6.0 | SPARK-11079 | 1797055dbf1d2fd7714d7c65c8d2efde2f15efc1#diff-05133dfc4bfdb6a27aa092d86ce24866 | spark.rpc.numRetries | 1.4.0 | SPARK-6490 | 8136810dfad12008ac300116df7bc8448740f1ae#diff-529fc5c06b9731c1fbda6f3db60b16aa | spark.rpc.retry.wait | 1.4.0 | SPARK-6490 | 8136810dfad12008ac300116df7bc8448740f1ae#diff-529fc5c06b9731c1fbda6f3db60b16aa | ### Why are the changes needed? Supplemental configuration version information. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Exists UT This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589945810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23565/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589945807 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
SparkQA commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589946380 **[Test build #118815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118815/testReport)** for PR 27674 at commit [`da39775`](https://github.com/apache/spark/commit/da39775c31b489b950cdd6db4b539517c58558f8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mgaido91 commented on issue #27629: [SPARK-28067][SQL]Fix incorrect results during aggregate sum for decimal overflow by throwing exception
mgaido91 commented on issue #27629: [SPARK-28067][SQL]Fix incorrect results during aggregate sum for decimal overflow by throwing exception URL: https://github.com/apache/spark/pull/27629#issuecomment-589947439 well' in this PR you are changing the logical plan, that's weird that the 2 executions mode return different results and we have to fix the plan for this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589945807 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589945810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23565/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
SparkQA commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948638 **[Test build #118816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118816/testReport)** for PR 22038 at commit [`b1958dd`](https://github.com/apache/spark/commit/b1958dd9a7a83ffb6576f04b13e210ae975f8263). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948753 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948759 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23566/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948753 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948759 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23566/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
wangyum commented on a change in pull request #22038: [SPARK-25056][SQL] Unify
the InConversion and BinaryComparison behavior
URL: https://github.com/apache/spark/pull/22038#discussion_r382907785
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
##
@@ -491,9 +491,13 @@ object TypeCoercion {
i
}
- case i @ In(a, b) if b.exists(_.dataType != a.dataType) =>
-findWiderCommonType(i.children.map(_.dataType)) match {
- case Some(finalDataType) => i.withNewChildren(i.children.map(Cast(_,
finalDataType)))
+ case i @ In(value, list) if list.exists(_.dataType != value.dataType) =>
Review comment:
Added a new config:
`spark.sql.legacy.inPredicateFollowBinaryComparisonTypeCoercion`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] peter-toth opened a new pull request #27675: [SPARK-30870][SQL] Fix nested column aliasing
peter-toth opened a new pull request #27675: [SPARK-30870][SQL] Fix nested
column aliasing
URL: https://github.com/apache/spark/pull/27675
### What changes were proposed in this pull request?
This PR fixes a bug in nested column aliasing by taking into account the
data type of the referenced nested field to calculate the number of extracted
columns. After this PR this query runs without issues:
```
SELECT explodedvalue.*
FROM VALUES array(named_struct('nested', named_struct('a', 1, 'b', 2))) AS
(value)
LATERAL VIEW explode(value) AS explodedvalue
```
This is a regression from Spark 2.4.
### Why are the changes needed?
To fix a bug.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Added new UT.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950445 **[Test build #118817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118817/testReport)** for PR 27675 at commit [`b09e19b`](https://github.com/apache/spark/commit/b09e19b6711b1d84774c4f6f22c51ce640ef2f72). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950543 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950545 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23567/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950543 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950545 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23567/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
SparkQA removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589934696 **[Test build #118813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118813/testReport)** for PR 27499 at commit [`096ce42`](https://github.com/apache/spark/commit/096ce420f8e5d04dac1b88e46c53820ace857507). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
SparkQA commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589953255 **[Test build #118813 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118813/testReport)** for PR 27499 at commit [`096ce42`](https://github.com/apache/spark/commit/096ce420f8e5d04dac1b88e46c53820ace857507). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589953403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118813/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589953399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589953403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118813/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
AmplabJenkins removed a comment on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589953399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
SparkQA commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589957172 **[Test build #118815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118815/testReport)** for PR 27674 at commit [`da39775`](https://github.com/apache/spark/commit/da39775c31b489b950cdd6db4b539517c58558f8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
SparkQA removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589946380 **[Test build #118815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118815/testReport)** for PR 27674 at commit [`da39775`](https://github.com/apache/spark/commit/da39775c31b489b950cdd6db4b539517c58558f8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589957371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118815/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins removed a comment on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589957370 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589957370 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network
AmplabJenkins commented on issue #27674: [SPARK-30888][CORE][DOC] Add version information to the configuration of Network URL: https://github.com/apache/spark/pull/27674#issuecomment-589957371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118815/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
SparkQA removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589938369 **[Test build #118814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118814/testReport)** for PR 27672 at commit [`a05af57`](https://github.com/apache/spark/commit/a05af57179adc2b0812841e71d04b9c818665d84). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
SparkQA commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589957710 **[Test build #118814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118814/testReport)** for PR 27672 at commit [`a05af57`](https://github.com/apache/spark/commit/a05af57179adc2b0812841e71d04b9c818665d84). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589957855 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118814/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589957855 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118814/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins removed a comment on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589957853 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations
AmplabJenkins commented on issue #27672: [SPARK-30919][SQL] Make interval multiply and divide's overflow behavior consistent with other operations URL: https://github.com/apache/spark/pull/27672#issuecomment-589957853 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589967876 **[Test build #118817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118817/testReport)** for PR 27675 at commit [`b09e19b`](https://github.com/apache/spark/commit/b09e19b6711b1d84774c4f6f22c51ce640ef2f72). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589950445 **[Test build #118817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118817/testReport)** for PR 27675 at commit [`b09e19b`](https://github.com/apache/spark/commit/b09e19b6711b1d84774c4f6f22c51ce640ef2f72). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589967940 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589967941 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118817/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589967940 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589967941 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118817/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
SparkQA removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589948638 **[Test build #118816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118816/testReport)** for PR 22038 at commit [`b1958dd`](https://github.com/apache/spark/commit/b1958dd9a7a83ffb6576f04b13e210ae975f8263). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
SparkQA commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589968531 **[Test build #118816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118816/testReport)** for PR 22038 at commit [`b1958dd`](https://github.com/apache/spark/commit/b1958dd9a7a83ffb6576f04b13e210ae975f8263). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589968719 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins commented on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589968722 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118816/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589968722 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118816/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior
AmplabJenkins removed a comment on issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComparison behavior URL: https://github.com/apache/spark/pull/22038#issuecomment-589968719 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] PedroCorreiaLuis commented on a change in pull request #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
PedroCorreiaLuis commented on a change in pull request #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#discussion_r382924316 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ## @@ -1901,6 +1908,7 @@ case class NestedStruct(f: ClassData) case class DeepNestedStruct(f: NestedStruct) case class InvalidInJava(`abstract`: Int) +case class InvalidInJavaForNumber(`1something`: String) Review comment: ```suggestion private[sql] case class InvalidInJavaForNumber(`1something`: String) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
SparkQA commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589972876 **[Test build #118818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118818/testReport)** for PR 26319 at commit [`6495368`](https://github.com/apache/spark/commit/64953685796f0294a0c2cfb1e5a1bfcb800b1c46). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973018 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973020 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23568/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973018 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973020 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23568/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973974 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118818/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
SparkQA removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589972876 **[Test build #118818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118818/testReport)** for PR 26319 at commit [`6495368`](https://github.com/apache/spark/commit/64953685796f0294a0c2cfb1e5a1bfcb800b1c46). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973974 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
SparkQA commented on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973968 **[Test build #118818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118818/testReport)** for PR 26319 at commit [`6495368`](https://github.com/apache/spark/commit/64953685796f0294a0c2cfb1e5a1bfcb800b1c46). * This patch **fails to generate documentation**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where…
AmplabJenkins removed a comment on issue #26319: [SPARK-29594][SQL] Create a Dataset from a Sequence of Case class where… URL: https://github.com/apache/spark/pull/26319#issuecomment-589973977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118818/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression
viirya commented on issue #27499: [SPARK-30590][SQL] Untyped select API cannot take typed column expression URL: https://github.com/apache/spark/pull/27499#issuecomment-589986070 also cc @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] peter-toth commented on a change in pull request #27675: [SPARK-30870][SQL] Fix nested column aliasing
peter-toth commented on a change in pull request #27675: [SPARK-30870][SQL] Fix
nested column aliasing
URL: https://github.com/apache/spark/pull/27675#discussion_r382934390
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala
##
@@ -215,12 +215,7 @@ class NestedColumnAliasingSuite extends SchemaPruningTest
{
val optimized = Optimize.execute(query)
-val expected = nestedRelation
- .select(GetStructField('a, 0, Some("b")))
- .limit(5)
- .analyze
-
-comparePlans(optimized, expected)
+comparePlans(optimized, query)
Review comment:
Since we need the whole structure, why we did expect the local relation to
be column pruned?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989062 **[Test build #118819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118819/testReport)** for PR 27675 at commit [`5a51b94`](https://github.com/apache/spark/commit/5a51b9472f5dfbf99ef6f8d6c7151618157c446e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989176 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989182 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23569/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989176 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989182 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23569/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] peter-toth commented on a change in pull request #27675: [SPARK-30870][SQL] Fix nested column aliasing
peter-toth commented on a change in pull request #27675: [SPARK-30870][SQL] Fix
nested column aliasing
URL: https://github.com/apache/spark/pull/27675#discussion_r382934390
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala
##
@@ -215,12 +215,7 @@ class NestedColumnAliasingSuite extends SchemaPruningTest
{
val optimized = Optimize.execute(query)
-val expected = nestedRelation
- .select(GetStructField('a, 0, Some("b")))
- .limit(5)
- .analyze
-
-comparePlans(optimized, expected)
+comparePlans(optimized, query)
Review comment:
Since we need the whole structure, why we expected the local relation to be
column pruned?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk opened a new pull request #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
MaxGekk opened a new pull request #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce any user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
SparkQA commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676#issuecomment-589994870 **[Test build #118820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118820/testReport)** for PR 27676 at commit [`561911a`](https://github.com/apache/spark/commit/561911a94a995426967e0e3014284b315de9ce56). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
AmplabJenkins commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676#issuecomment-589995074 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
AmplabJenkins commented on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676#issuecomment-589995077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23570/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
AmplabJenkins removed a comment on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676#issuecomment-589995074 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds
AmplabJenkins removed a comment on issue #27676: [WIP][SQL] Prevent overflow in conversions of milliseconds to/from microseconds URL: https://github.com/apache/spark/pull/27676#issuecomment-589995077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23570/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-589989062 **[Test build #118819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118819/testReport)** for PR 27675 at commit [`5a51b94`](https://github.com/apache/spark/commit/5a51b9472f5dfbf99ef6f8d6c7151618157c446e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
SparkQA commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-590008439 **[Test build #118819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118819/testReport)** for PR 27675 at commit [`5a51b94`](https://github.com/apache/spark/commit/5a51b9472f5dfbf99ef6f8d6c7151618157c446e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-590008580 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins commented on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-590008582 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118819/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-590008582 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118819/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing
AmplabJenkins removed a comment on issue #27675: [SPARK-30870][SQL] Fix nested column aliasing URL: https://github.com/apache/spark/pull/27675#issuecomment-590008580 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-590008928 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-590008931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23571/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-590008928 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-590008931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23571/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-590009309 **[Test build #118821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118821/testReport)** for PR 27570 at commit [`653b0dc`](https://github.com/apache/spark/commit/653b0dc33a56ebcd8a5c11c9eedb92a94e5d7a58). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] nandini57 commented on issue #27056: [SPARK-27217][SQL] Nested schema pruning with Aggregation
nandini57 commented on issue #27056: [SPARK-27217][SQL] Nested schema pruning with Aggregation URL: https://github.com/apache/spark/pull/27056#issuecomment-590009635 Hi Spark Team, I am inclined to add this change as a custom logical rule by copying over the Aggregate Nesting object in my spark project.We are using Spark 2.3 version and no immediate plans to move over to 3.x. Do you think it is a viable approach ?Any guidance much appreciated This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-590009797 **[Test build #118823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118823/testReport)** for PR 27571 at commit [`19e6126`](https://github.com/apache/spark/commit/19e6126ecd25c98dc0804b78c718d3e0517e15ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
