Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
No problem, thanks @gatorsmile !
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
@BryanCutler Thank you for all your hard work. We really appreciate it!
Merged to master.
---
-
To unsubscribe,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
@ueshin do you think this is ready to merge? cc @gatorsmile @cloud-fan for
another look
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83087/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #83087 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83087/testReport)**
for PR 18664 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #83087 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83087/testReport)**
for PR 18664 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83050/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #83050 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83050/testReport)**
for PR 18664 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
>Before we merging this PR, could anybody submit a PR for documenting this
issue? Then, we can get more feedbacks from the others.
@gatorsmile I started a wip at #19575 for adding
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #83050 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83050/testReport)**
for PR 18664 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
@ueshin I have the requested changes done, just wondering if you could
clarify https://github.com/apache/spark/pull/18664#discussion_r146472494 before
I push them, thanks!
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82997/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82997 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82997/testReport)**
for PR 18664 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
I cleaned up some of the timestamp conversion code and added a test for a
`pandas_udf` that returns a `DateType` which is currently causing an error.
see
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82997 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82997/testReport)**
for PR 18664 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
BTW, we might need to resolve the comments in
https://github.com/apache/spark/pull/18933 and merge that PR first.
---
-
To
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
Before we merging this PR, could anybody submit a PR for documenting this
issue? Then, we can get more feedbacks from the others.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82915/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82915 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82915/testReport)**
for PR 18664 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82915 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82915/testReport)**
for PR 18664 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
Thanks @cloud-fan , so is everyone in agreement with the current behavior
of this PR? It sounds like we will probably need a follow up discussion to
address some details, but with Arrow
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18664
LGTM, my only concern is, the `timezoneId` parameter should not be a
`Option[String]` but just a `String`.
---
-
To
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/18664
> Do you think the behavior of current toPandas() without Arrow is a bug
and should be fixed?
I personally consider it a bug. Having the same code return different
results depending on your
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/18664
@cloud-fan Unfortunately, tz-naive Arrow and the current `toPandas()`
without Arrow make different result.
For example, let's say we use `"America/Los_Angeles"` as session timezone
and have
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
> BTW if Spark supports timestamp with timezone, then the timezone carried
by the timestamp value should be picked
+1 for Spark supporting this, I think it would make things easier
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
Thanks @cloud-fan. To sum up the current state of this PR: timestamps are
written to Arrow as values from unix epoch, in UTC (same long value as Spark
SQL internal storage) and given a timezone
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18664
Sorry for joining this discussion so late. My 2 cents:
Spark SQL only has timestamp without timezone, and internally we use a java
long to store the value of a timestamp. This is OK as
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
Will take another look tomorrow. Thanks for your work! @BryanCutler
---
-
To unsubscribe, e-mail:
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/18664
I'd cc @gatorsmile, @cloud-fan for final check.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18664
LGTM too
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82862/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82862 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82862/testReport)**
for PR 18664 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #82862 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82862/testReport)**
for PR 18664 at commit
39 matches
Mail list logo
