Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/18664 I agree with Bryan. I think we might want to rethink the assumption that toPandas result with arrow / without arrow should be 100% the same. For instance, non-Arrow doesn't respect session local timezone, and if for compatible reasons that we cannot fix this in the non-Arrow version, then let's fix it in the Arrow version and document the difference. IMHO, keeping a new feature bug compatible of the existing feature is not necessarily, fixing non ideal behavior in the new feature it provides a migration path off the buggy behavior of existing feature. On Thu, Oct 5, 2017 at 7:34 PM Bryan Cutler <notificati...@github.com> wrote: > @ueshin <https://github.com/ueshin> @HyukjinKwon > <https://github.com/hyukjinkwon> , I think it would be critical for users > to have timestamps working for Arrow. Just to recap, the remaining issue > here was that toPandas() without Arrow does not have timestamps with a > timezone. Is it possible we can document that difference and not hold this > up for fixing the case without Arrow? Arrow is still disabled by default, > so the default behavior of toPandas() does not change. > > â > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/18664#issuecomment-334620292>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAwbrGUKC2aM-9gfFvLlw_SN2SUwrFQQks5spWehgaJpZM4Oateu> > . >
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org