[GitHub] [spark] viirya commented on issue #26159: [SPARK-29506][SQL] Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table
viirya commented on issue #26159: [SPARK-29506][SQL] Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table URL: https://github.com/apache/spark/pull/26159#issuecomment-557430864 @cloud-fan Would like to hear if you have other comments? If you still have concern on this, I will close this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #26461: [SPARK-29831][SQL] Scan Hive partitioned table should not dramatically increase data parallelism
viirya commented on issue #26461: [SPARK-29831][SQL] Scan Hive partitioned table should not dramatically increase data parallelism URL: https://github.com/apache/spark/pull/26461#issuecomment-557429259 For now we will take another approach to this issue. Hive scan has a few other questions like predicate pushdown, schema pruning. Although I think this is a real problem, I may not have time to follow up on this. Thus close it first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya closed pull request #26461: [SPARK-29831][SQL] Scan Hive partitioned table should not dramatically increase data parallelism
viirya closed pull request #26461: [SPARK-29831][SQL] Scan Hive partitioned table should not dramatically increase data parallelism URL: https://github.com/apache/spark/pull/26461 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
dilipbiswal commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557425164 cc @srowen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher`
SparkQA commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher` URL: https://github.com/apache/spark/pull/26609#issuecomment-557424840 **[Test build #114281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114281/testReport)** for PR 26609 at commit [`fb79d0a`](https://github.com/apache/spark/commit/fb79d0ac15c6f572ec3d26fde4e73727cff95a9d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] normanmaurer commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher`
normanmaurer commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher` URL: https://github.com/apache/spark/pull/26609#issuecomment-557423085 @vanzin @dongjoon-hyun addressed all + squashed and adjusted commit message to reflect the final patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher`
SparkQA commented on issue #26609: [SPARK-29971] Fix multiple possible buffer leaks in `TransportFrameDecoder/TransportCipher` URL: https://github.com/apache/spark/pull/26609#issuecomment-557422801 **[Test build #114280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114280/testReport)** for PR 26609 at commit [`d8d2bbe`](https://github.com/apache/spark/commit/d8d2bbedd5a09209ec305d528d0e7bed63d4621a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support
viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL]
Arrow MapType support
URL: https://github.com/apache/spark/pull/26512#discussion_r349460054
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
##
@@ -343,3 +348,36 @@ private[arrow] class StructWriter(
children.foreach(_.reset())
}
}
+
+private[arrow] class MapWriter(val valueVector: MapVector,
+ val keyWriter: ArrowFieldWriter,
+ val valueWriter: ArrowFieldWriter) extends
ArrowFieldWriter {
+
+ override def setNull(): Unit = {}
Review comment:
can't we set null for a map vector?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557421477 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557421477 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
SparkQA removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557418870 **[Test build #114279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114279/testReport)** for PR 26635 at commit [`dab073a`](https://github.com/apache/spark/commit/dab073accb17d4b85515e9f43a2b9ee1643a7e74). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
SparkQA commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557421395 **[Test build #114279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114279/testReport)** for PR 26635 at commit [`dab073a`](https://github.com/apache/spark/commit/dab073accb17d4b85515e9f43a2b9ee1643a7e74). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557421485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114279/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557421485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114279/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557419244 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19122/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins removed a comment on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557419235 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557419244 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19122/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
AmplabJenkins commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557419235 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
SparkQA commented on issue #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635#issuecomment-557418870 **[Test build #114279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114279/testReport)** for PR 26635 at commit [`dab073a`](https://github.com/apache/spark/commit/dab073accb17d4b85515e9f43a2b9ee1643a7e74). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal opened a new pull request #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference
dilipbiswal opened a new pull request #26635: [SPARK-28812][SQL][DOC] Document SHOW PARTITIONS in SQL Reference URL: https://github.com/apache/spark/pull/26635 ### What changes were proposed in this pull request? Document SHOW PARTITIONS statement in SQL Reference Guide. ### Why are the changes needed? Currently Spark lacks documentation on the supported SQL constructs causing confusion among users who sometimes have to look at the code to understand the usage. This is aimed at addressing this issue. ### Does this PR introduce any user-facing change? Yes. **Before** **After**    This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on issue #26141: [SPARK-29492][SQL]Reset HiveSession's SessionState conf's ClassLoader when sync mode
AngersZh commented on issue #26141: [SPARK-29492][SQL]Reset HiveSession's SessionState conf's ClassLoader when sync mode URL: https://github.com/apache/spark/pull/26141#issuecomment-557416769 ping @wangyum @LantaoJin @sandeep-katta This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557416327 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
AmplabJenkins removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557416332 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114270/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557416332 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114270/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
AmplabJenkins commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557416327 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
AmplabJenkins removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557415712 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
SparkQA removed a comment on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557379726 **[Test build #114270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114270/testReport)** for PR 26633 at commit [`bb91814`](https://github.com/apache/spark/commit/bb918144e2671193970a978241f2334107c90fe4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
AmplabJenkins removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557415721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114265/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
AmplabJenkins commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557415721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114265/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
SparkQA commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-557415783 **[Test build #114270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114270/testReport)** for PR 26633 at commit [`bb91814`](https://github.com/apache/spark/commit/bb918144e2671193970a978241f2334107c90fe4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
AmplabJenkins commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557415712 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
SparkQA removed a comment on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557366644 **[Test build #114265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114265/testReport)** for PR 26617 at commit [`f0a3e4d`](https://github.com/apache/spark/commit/f0a3e4d7872d2c7fa08e978f94e2ebea4da46302). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces
SparkQA commented on issue #26617: [SPARK-29979][SQL]Add basic/reserved property key constants in Table and SupportsNamespaces URL: https://github.com/apache/spark/pull/26617#issuecomment-557415199 **[Test build #114265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114265/testReport)** for PR 26617 at commit [`f0a3e4d`](https://github.com/apache/spark/commit/f0a3e4d7872d2c7fa08e978f94e2ebea4da46302). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
MaxGekk commented on a change in pull request #26622: [SPARK-28023][SQL] Add
trim logic in UTF8String's toInt/toLong to make it consistent with other
string-numeric casting
URL: https://github.com/apache/spark/pull/26622#discussion_r349453822
##
File path:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
##
@@ -1077,26 +1077,28 @@ public UTF8String translate(Map
dict) {
* @return true if the parsing was successful else false
*/
public boolean toLong(LongWrapper toLongResult) {
-if (numBytes == 0) {
- return false;
-}
+int offset = 0;
+while (offset < this.numBytes && getByte(offset) <= ' ') offset++;
+if (offset == this.numBytes) return false;
-byte b = getByte(0);
+int end = this.numBytes - 1;
+while (end > offset && getByte(end) <= ' ') end--;
Review comment:
I see
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
MaxGekk commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413553 @cloud-fan I ran `ThriftServerQueryTestSuite`. It failed by the the same reasons. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk edited a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
MaxGekk edited a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413553 @cloud-fan I ran `ThriftServerQueryTestSuite` locally. It failed by the the same reasons. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
AmplabJenkins removed a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413108 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
AmplabJenkins removed a comment on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19121/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
AmplabJenkins commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19121/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
AmplabJenkins commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557413108 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557412659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114269/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals
SparkQA commented on issue #26473: [SPARK-29864][SPARK-29920][SQL] Strict parsing of day-time strings to intervals URL: https://github.com/apache/spark/pull/26473#issuecomment-557412766 **[Test build #114278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114278/testReport)** for PR 26473 at commit [`73ef32f`](https://github.com/apache/spark/commit/73ef32fa497bebf7982d92146657ff9c9d77e5f0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557412650 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557412650 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557412659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114269/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
SparkQA commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557412204 **[Test build #114269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114269/testReport)** for PR 26626 at commit [`79abc93`](https://github.com/apache/spark/commit/79abc93886e4ea78295d3d8c014a73fe3cb38ebb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
SparkQA removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557376674 **[Test build #114269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114269/testReport)** for PR 26626 at commit [`79abc93`](https://github.com/apache/spark/commit/79abc93886e4ea78295d3d8c014a73fe3cb38ebb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
AmplabJenkins removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557411836 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114264/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
AmplabJenkins commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557411836 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114264/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
AmplabJenkins commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557411827 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
AmplabJenkins removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557411827 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
yaooqinn commented on a change in pull request #26622: [SPARK-28023][SQL] Add
trim logic in UTF8String's toInt/toLong to make it consistent with other
string-numeric casting
URL: https://github.com/apache/spark/pull/26622#discussion_r349450963
##
File path:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
##
@@ -1077,26 +1077,28 @@ public UTF8String translate(Map
dict) {
* @return true if the parsing was successful else false
*/
public boolean toLong(LongWrapper toLongResult) {
-if (numBytes == 0) {
- return false;
-}
+int offset = 0;
+while (offset < this.numBytes && getByte(offset) <= ' ') offset++;
+if (offset == this.numBytes) return false;
-byte b = getByte(0);
+int end = this.numBytes - 1;
+while (end > offset && getByte(end) <= ' ') end--;
Review comment:
Hm, I guess not, how do you know the ' ' is in the middle or end?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
SparkQA removed a comment on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557364833 **[Test build #114264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114264/testReport)** for PR 26625 at commit [`1d4a769`](https://github.com/apache/spark/commit/1d4a769ad320c395e9c2e11361c3c57eceddd2cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader
SparkQA commented on issue #26625: [SPARK-29893][SQL][followup] code cleanup for local shuffle reader URL: https://github.com/apache/spark/pull/26625#issuecomment-557411438 **[Test build #114264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114264/testReport)** for PR 26625 at commit [`1d4a769`](https://github.com/apache/spark/commit/1d4a769ad320c395e9c2e11361c3c57eceddd2cb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support
viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL]
Arrow MapType support
URL: https://github.com/apache/spark/pull/26512#discussion_r349450540
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
##
@@ -343,3 +348,36 @@ private[arrow] class StructWriter(
children.foreach(_.reset())
}
}
+
+private[arrow] class MapWriter(val valueVector: MapVector,
+ val keyWriter: ArrowFieldWriter,
+ val valueWriter: ArrowFieldWriter) extends
ArrowFieldWriter {
Review comment:
please follow usual indent style.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] fuwhu closed pull request #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxMetadataValueStringLength'
fuwhu closed pull request #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxMetadataValueStringLength' URL: https://github.com/apache/spark/pull/26356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] fuwhu commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxMetadataValueStringLength'
fuwhu commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxMetadataValueStringLength' URL: https://github.com/apache/spark/pull/26356#issuecomment-557410063 I'd like to close this PR for now, will reopen if got better idea to resolve it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557409647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557409641 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557409641 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557409647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL] Arrow MapType support
viirya commented on a change in pull request #26512: [WIP][SPARK-29493][SQL]
Arrow MapType support
URL: https://github.com/apache/spark/pull/26512#discussion_r349449391
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala
##
@@ -88,6 +90,16 @@ object ArrowUtils {
fields.map { field =>
toArrowField(field.name, field.dataType, field.nullable,
timeZoneId)
}.toSeq.asJava)
+ case MapType(keyType, valueType, valueContainsNull) =>
+val mapType = new FieldType(valueContainsNull, new
ArrowType.Map(false), null)
Review comment:
`valueContainsNull` is for map values have null or not. For FieldType here,
I think it is like ArrayType and StructType cases, we should use given nullable
parameter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
AmplabJenkins removed a comment on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns URL: https://github.com/apache/spark/pull/26593#issuecomment-557409066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
SparkQA removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557363126 **[Test build #114263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114263/testReport)** for PR 26590 at commit [`8d6d08b`](https://github.com/apache/spark/commit/8d6d08b0224a6e98fd96099c52bfed33d6afb733). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557409181 **[Test build #114263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114263/testReport)** for PR 26590 at commit [`8d6d08b`](https://github.com/apache/spark/commit/8d6d08b0224a6e98fd96099c52bfed33d6afb733). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
AmplabJenkins removed a comment on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns URL: https://github.com/apache/spark/pull/26593#issuecomment-557409075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19120/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
AmplabJenkins commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns URL: https://github.com/apache/spark/pull/26593#issuecomment-557409075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19120/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
AmplabJenkins commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns URL: https://github.com/apache/spark/pull/26593#issuecomment-557409066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] advancedxy commented on issue #25919: [WIP][SPARK-15616][SQL] Hive table supports partition pruning in JoinSelection
advancedxy commented on issue #25919: [WIP][SPARK-15616][SQL] Hive table supports partition pruning in JoinSelection URL: https://github.com/apache/spark/pull/25919#issuecomment-557409288 Gently ping @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
SparkQA commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557408818 **[Test build #114276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114276/testReport)** for PR 26626 at commit [`773eb4b`](https://github.com/apache/spark/commit/773eb4b8678925bf5d78b4538583496775873bb6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
SparkQA commented on issue #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns URL: https://github.com/apache/spark/pull/26593#issuecomment-557408820 **[Test build #114277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114277/testReport)** for PR 26593 at commit [`702897c`](https://github.com/apache/spark/commit/702897c9792f100b08d7c9b4b7de7c455ad278ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #26593: [SPARK-29890][SQL] DataFrameNaFunctions.fill should handle duplicate columns
imback82 commented on a change in pull request #26593: [SPARK-29890][SQL]
DataFrameNaFunctions.fill should handle duplicate columns
URL: https://github.com/apache/spark/pull/26593#discussion_r349448233
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala
##
@@ -468,12 +477,26 @@ final class DataFrameNaFunctions private[sql](df:
DataFrame) {
s"Unsupported value type ${v.getClass.getName} ($v).")
}
+ private def toAttributes(cols: Seq[String]): Seq[Attribute] = {
+def resolve(colName: String) : Attribute = {
+ df.col(colName).named.toAttribute match {
+case a: Attribute => a
+case _ => throw new IllegalArgumentException(s"'$colName' is not a top
level column.")
Review comment:
great, thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557407430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19119/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557407345 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557407423 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins removed a comment on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557407351 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19118/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
AmplabJenkins commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557407430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19119/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
AmplabJenkins commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557407423 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
MaxGekk commented on a change in pull request #26622: [SPARK-28023][SQL] Add
trim logic in UTF8String's toInt/toLong to make it consistent with other
string-numeric casting
URL: https://github.com/apache/spark/pull/26622#discussion_r349447501
##
File path:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
##
@@ -1077,26 +1077,28 @@ public UTF8String translate(Map
dict) {
* @return true if the parsing was successful else false
*/
public boolean toLong(LongWrapper toLongResult) {
-if (numBytes == 0) {
- return false;
-}
+int offset = 0;
+while (offset < this.numBytes && getByte(offset) <= ' ') offset++;
+if (offset == this.numBytes) return false;
-byte b = getByte(0);
+int end = this.numBytes - 1;
+while (end > offset && getByte(end) <= ' ') end--;
Review comment:
You don't need to trim from the right explicitly here. Just break inside the
loop
https://github.com/apache/spark/pull/26622/files#diff-d2b5337b91f684b9e7fd5cc101e93fc8R1104
if b == ' '
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557407345 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
AmplabJenkins commented on issue #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String URL: https://github.com/apache/spark/pull/26626#issuecomment-557407351 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19118/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #26626: [SPARK-29986][SQL] Introduce java like string trim to UTF8String
yaooqinn commented on a change in pull request #26626: [SPARK-29986][SQL]
Introduce java like string trim to UTF8String
URL: https://github.com/apache/spark/pull/26626#discussion_r349447132
##
File path:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java
##
@@ -553,6 +553,34 @@ public UTF8String trim() {
return copyUTF8String(s, e);
}
+ /**
+ * Trims whitespaces (<= ASCII 32) from both ends of this string.
+ *
+ * Note that, this method is the same as java's {@link String#trim}, and
different from
+ * {@link UTF8String#trim()} which only remove only spaces(= ASCII 32) from
both ends.
+ *
+ * @return A UTF8String whose value is this UTF8String, with any leading and
trailing white
+ * space removed, or this UTF8String if it has no leading or trailing
whitespace.
+ *
+ */
+ public UTF8String trimAll() {
+int s = 0;
+// skip all of the whitespaces (<=0x20) in the left side
+while (s < this.numBytes && getByte(s) <= 0x20) s++;
Review comment:
maybe also replace them in method trim? I don't know if whether ok or not I
do so in this pr
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
SparkQA commented on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557407017 **[Test build #114275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114275/testReport)** for PR 26622 at commit [`d5c2a40`](https://github.com/apache/spark/commit/d5c2a40b5bcbb095dca811b568255c5f30539fe7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive
AmplabJenkins commented on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive URL: https://github.com/apache/spark/pull/26634#issuecomment-557405483 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557405355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive
AmplabJenkins removed a comment on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive URL: https://github.com/apache/spark/pull/26634#issuecomment-557405209 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557405353 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557405353 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive
AmplabJenkins commented on issue #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive URL: https://github.com/apache/spark/pull/26634#issuecomment-557405209 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
AmplabJenkins commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557405355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
SparkQA removed a comment on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557359637 **[Test build #114262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114262/testReport)** for PR 26590 at commit [`f9dc1a4`](https://github.com/apache/spark/commit/f9dc1a4f3f12f04f216352d543727ceeac3015c8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink
SparkQA commented on issue #26590: [SPARK-29953][SS] Don't clean up source files for FileStreamSource if the files belong to the output of FileStreamSink URL: https://github.com/apache/spark/pull/26590#issuecomment-557404983 **[Test build #114262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114262/testReport)** for PR 26590 at commit [`f9dc1a4`](https://github.com/apache/spark/commit/f9dc1a4f3f12f04f216352d543727ceeac3015c8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ulysses-you opened a new pull request #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive
ulysses-you opened a new pull request #26634: [SPARK-29996][SQL] Add v2 command exec check in adaptive URL: https://github.com/apache/spark/pull/26634 ### What changes were proposed in this pull request? AdaptivePlan should also ignore v2 command exec plan like v1. ### Why are the changes needed? Avoid logging. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
AmplabJenkins commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557402021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114268/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
AmplabJenkins removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557402021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114268/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
AmplabJenkins removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557402015 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
AmplabJenkins commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557402015 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
SparkQA commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557401575 **[Test build #114268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114268/testReport)** for PR 26201 at commit [`e30af98`](https://github.com/apache/spark/commit/e30af988f6325e8d3ccd589ea28adc1357dc1e3d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
SparkQA removed a comment on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-557373332 **[Test build #114268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/114268/testReport)** for PR 26201 at commit [`e30af98`](https://github.com/apache/spark/commit/e30af988f6325e8d3ccd589ea28adc1357dc1e3d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting
AmplabJenkins removed a comment on issue #26622: [SPARK-28023][SQL] Add trim logic in UTF8String's toInt/toLong to make it consistent with other string-numeric casting URL: https://github.com/apache/spark/pull/26622#issuecomment-557400874 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
