On Wed, 7 Jul 2021 at 18:46, Jorge Cardoso Leitão <jorgecarlei...@gmail.com> wrote:
> Hi, > > AFAIK timezone is part of the spec. And for reference, the current spec (Schema flatbuffer file) for timestamp is at https://github.com/apache/arrow/blob/6c8d30ea82222fd2750b999840872d3f6cbdc8f8/format/Schema.fbs#L217-L247. > In Python, that would be [1] > > import pyarrow as pa > dt1 = pa.timestamp("ms", "+00:10") > dt2 = pa.timestamp("ms") > > arrow-rs is not very consistent with how it handles it. imo that is an > artifact of being currently difficult (API wise) to create an array with a > timezone, which have caused people to not use it much (and thus not > implement kernels with it / test it properly). > > I do not see how removing it would be compatible with the Arrow spec, > though. > > Best, > Jorge > > [1] https://arrow.apache.org/docs/python/generated/pyarrow.timestamp.html > > > > On Wed, Jul 7, 2021 at 6:37 PM Evan Chan <e...@urbanlogiq.com> wrote: > > > Hi folks, > > > > Some of us are having a discussion about a direction change for Rust > Arrow > > timestamp types, which current support both a resolution field (Ns, > Micros, > > Ms, Seconds) similar to the other language implementations, but also > > optionally a timezone string field. I believe the timezone field is > > unique to the Rust implementation, as I don’t find it in the C/C++ and > > Python docs. At the same time, in reality if the timezone field is non > > null, this is not well supported at all in the current code. Functions > > returning timestamps pretty much all return a null timezone, for example, > > and don’t allow the timezone to be specified. > > > > The proposal would be to eliminate the timezone field and bring the Rust > > Arrow timestamp type in line with that of the other language > > implementations, also simplifying implementation. It seems this is in > > line with direction of other projects (Parquet, Spark, and most DBs have > > timestamp types which do not have explicit timezones or are implicitly > UTC). > > > > Please feel free to see > > https://github.com/apache/arrow-datafusion/issues/686 < > > https://github.com/apache/arrow-datafusion/issues/686> > > (Or would it be better to discuss here in mailing list?) > > > > Cheers! > > Evan >
