You could store epoch offsets, but interpret them in the local timezone. E.g. (0, "America/New_York") could mean 1970-01-01 00:00:00 in the New York timezone. At least one nasty problem with that is ambiguous times, i.e. when the clock turns back on going from DST to ST, as well as invalid times (when the clock moves forwards, meaning some epoch offsets never occur).
On Wed, Jun 2, 2021 at 1:58 PM Joris Van den Bossche < jorisvandenboss...@gmail.com> wrote: > On Wed, 2 Jun 2021 at 13:56, Antoine Pitrou <anto...@python.org> wrote: > > > > > Hello, > > > > For the first time I notice this piece of information about the > > timestamp type: > > > > > /// * If the time zone is set to a valid value, values can be > > displayed as > > > /// "localized" to that time zone, even though the underlying > 64-bit > > > /// integers are identical to the same data stored in UTC. > Converting > > > /// between time zones is a metadata-only operation and does not > > change the > > > /// underlying values > > > > (from https://github.com/apache/arrow/blob/master/format/Schema.fbs#L223 > ) > > > > This seems rather weird to me: timestamps always convey a UTC timestamp > > value, optionally decorated with a local timezone? What is the > > motivation for such a representation? It is unlike other systems such > > as Python, where a timezone-aware timestamp really expresses a local > > time value, not a UTC time value. > > > > Just as reference: pandas uses the same model of storing UTC timestamps for > timezone-aware data (I think numpy also stored it as UTC, before they > removed support for it). And for example, I think also databases like > Postgresql store it as UTC internally, AFAIK. > The Python standard library datetime.datetime indeed stores localized > timestamps. But important difference is that Python actually stores the > year/month/day/hour/etc as separate values, so directly representing an > actual moment in time in a certain timezone. While I think what we store is > considered as "unix time"? (epoch since January 1st, 1970 at UTC) I am not > sure how you would store a timestamp in a certain timezone in this model. > > Some advantages of storing UTC that come to mind: it makes converting from > one timezone to another a trivial (metadata-only) operation, makes easier > to do timestamp comparisons across timezones, and it makes > timedelta-arithmetic easier. > > Joris > > > > Thank you, > > > > Antoine. > > >
