Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63917/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #63917 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63917/consoleFull)**
for PR 14279 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #63917 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63917/consoleFull)**
for PR 14279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63915/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #63915 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63915/consoleFull)**
for PR 14279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
Let me please rebase as it is now different a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #63915 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63915/consoleFull)**
for PR 14279 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
Sure, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14279
That'd be great. I will review it asap when you do it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
@rxin, Sure! I can add `dateFormat` option for both. Do you want me to do
this here (or follow-up)?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14279
@HyukjinKwon can we also allow users to set the output date/timestamp
format? This is a breaking change, and I really worry about that. With this
change and without making it configurable, there will
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14279
Out right now. Will take a look at this tonight or tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14279
OK, I'd like to merge this for master if there are no other comments. I'm
waiting more than usual to make sure
---
If your project is set up for it, you can reply to this email and have your
reply
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
@srowen I think it is okay as a separate PR.
I believe adding the things I said for a follow-up will mess up the change
and make this hard to be reviewed. Nevertheless, I don't mind
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14279
@HyukjinKwon are you OK with the current state of this PR? you mentioned a
few follow-ups and I'm wondering if you think they're important to include here.
@rxin how do you feel about it?
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
Yes, as shown in
https://github.com/apache/spark/pull/14279#issuecomment-236469454
(but we should still manually give the schema as inferring `DateType` and
`TimestampType` is not
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14279
Are we able to read the output back once written as JSON/CSV?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
Yes, it uses ISO 8601 by default and yes, it seems there is an issue for
parsing dates before 1970.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/14279
+1 to default formats that are unambiguous regarding timezone
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14279
I like this change. It does introduce a reasonably significant behavior
change in the output, even if the output is better. CC @rxin who brought that
up. The problem with changing CSV to match JSON
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
I might have to make a follow-up for..
- making `DatatypeConverter.parseDateTime` to be safe with early dates
- enabling `TimestampType` tests in `JsonHadoopFsRelationSuite`
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
I think it is okay. Just to make sure,
- **This writes date and timestamps fine**:
```scala
val path = "/tmp"
val date = java.sql.Date.valueOf("1970-01-01")
Github user deanchen commented on the issue:
https://github.com/apache/spark/pull/14279
@srowen ltgm, thanks for chiming in!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14279
@HyukjinKwon is this good to go as far as you're concerned? @deanchen I
assume you've reviewed and are OK with it.
---
If your project is set up for it, you can reply to this email and have your
Github user deanchen commented on the issue:
https://github.com/apache/spark/pull/14279
Would be great to get this in too. Currently using a hack where we iterate
through all the date columns at the end of our run and manually converting the
values to the string values formatted as
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62817/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62817 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62817/consoleFull)**
for PR 14279 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62817 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62817/consoleFull)**
for PR 14279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62642/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62642 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62642/consoleFull)**
for PR 14279 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62642 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62642/consoleFull)**
for PR 14279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62597/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62597 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62597/consoleFull)**
for PR 14279 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62597 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62597/consoleFull)**
for PR 14279 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
It seems due to https://issues.apache.org/jira/browse/LANG-982 I will look
into this deeper
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62584 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62584/consoleFull)**
for PR 14279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62584/
Test FAILed.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14279
cc @rxin, @srowen and @deanchen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14279
**[Test build #62584 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62584/consoleFull)**
for PR 14279 at commit
45 matches
Mail list logo
