[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22913 Sounds good, thanks @javierluraschi ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user javierluraschi commented on the issue: https://github.com/apache/spark/pull/22913 Yes, I believe that's the case. This change https://github.com/apache/arrow/pull/2887 maps R bindings from `POSIXct` to `timestamp` instead of `date`, looks like there is not need for this one, I'll close it and send a new PR if we find out something else is missing. Thanks everyone for your feedback, while not a change in Spark, ended up helping the Arrow project. Best. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22913 I'm a little against adding this because the Arrow Java Vectors used so far were done to match the internal data of Spark, to keep things simple and avoid lots of conversions on the Java side. Conversions to supported types are being done before reading the data in Java. For instance, there are lots of timestamp types for other time units, but we only accept microseconds with tz in Java (to match Spark) and do any necessary conversions in Python before writing the Arrow data. Can this also be done in R? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98482/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98482/testReport)** for PR 22913 at commit [`f809942`](https://github.com/apache/spark/commit/f809942de6c241cc7b499c19d0250185ebe26122). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98482/testReport)** for PR 22913 at commit [`f809942`](https://github.com/apache/spark/commit/f809942de6c241cc7b499c19d0250185ebe26122). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22913 We map `POSIXct` to `TimestampType` in R. I think it makes more sense to not support this. Historically Parquet also choses to better not support. For instance, `UINT_*`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98455/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98455 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98455/testReport)** for PR 22913 at commit [`3afb870`](https://github.com/apache/spark/commit/3afb8708c0394368a9435a7911201de31143f41e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98454/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98454/testReport)** for PR 22913 at commit [`2c14694`](https://github.com/apache/spark/commit/2c146941adb294ec9c5acc93cf55108e88075ad2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22913 Also cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98455 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98455/testReport)** for PR 22913 at commit [`3afb870`](https://github.com/apache/spark/commit/3afb8708c0394368a9435a7911201de31143f41e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98454/testReport)** for PR 22913 at commit [`2c14694`](https://github.com/apache/spark/commit/2c146941adb294ec9c5acc93cf55108e88075ad2). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98418/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98418/testReport)** for PR 22913 at commit [`f809942`](https://github.com/apache/spark/commit/f809942de6c241cc7b499c19d0250185ebe26122). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22913 Can we add a test in `ArrowUtilsSuite.scala`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22913 **[Test build #98418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98418/testReport)** for PR 22913 at commit [`f809942`](https://github.com/apache/spark/commit/f809942de6c241cc7b499c19d0250185ebe26122). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22913 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22913 cc @BryanCutler --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22913: [SPARK-25902][SQL] Add support for dates with millisecon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22913 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
