[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 @nbeyer Thanks for bearing with me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon Yeah, I agree, this PR doesn't really apply anymore. I'll work up a different JIRA and PR. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 I mean the problem in the JIRA is not reproduced in the master branch and therefore I believe we need another JIRA to describe the support for other time formats as the same one as casting opera

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 @nbeyer Thanks for your investigation. I think that sounds reasonable though I think it might be arguable because adding more cases virtually means more time and computation to parse/infer schem

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon Based on my further reading of the code, I'd like to suggest that add a deprecation to the stringToTime method and then update the stringToTimestamp method, specifically here https://gi

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 @nbeyer I manually checked the method call before and it seems that is currently only called when reading/inferring CSV/JSON (but I'd like you confirm this to double check in case). --- If you

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon I'll have to try out the 2.0.1/master changes. Is this DateTimeUtils method no longer used anywhere then? The other scenario where I ran into trouble was using 'cast' on Column class.

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 FYI, we backported this to branch 2.0 too. So this will be fixed from 2.0.1 https://github.com/apache/spark/pull/14799. --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 cc @srowen who was in the JIRA too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15147 To continue the discussion of JIRA, I think the issue you faced is to read those in CSV? Whether it is intended or not in `FastDateFormat`, the default pattern `"-MM-dd'T'HH:mm:ss.S

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15147 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat