I've dual-published the article on github and medium under the title 'A comprehensive guide to NumPy data types'.
Thank you all for your help and happy New Year! Best regards, Lev On Sun, Jan 2, 2022 at 12:31 AM Stefano Miccoli <stefano.micc...@polimi.it> wrote: > First of all, happy new 2022 UTC year! > > Let my add just a very brief note to this discussion: I opened > https://github.com/numpy/numpy/issues/20675 which addresses the > shortcomings of the current documentation, which is in my opinion not > sufficiently explicit in stating the datetime64 semantics. It is true that > these are largely consistent with python ‘datetime.datetime’, but ‘explicit > is better than implicit’. If nobody objects I will then open a doc-only PR > adding a very short paragraph to the docs trying to explain the points > discussed here. > > As what regards how much time UTC gained from 1970-01-01 up to today, you > are right, it’s about 29 s. The UTC timescale was officially born in 1963 > but it can be traced back at least up to 1956/1958, see > https://github.com/skyfielders/python-skyfield/issues/679 where this is > discussed with reference to the timescales implemented in python-skyfield. > > Stefano > > On 31 Dec 2021, at 13:27, numpy-discussion-requ...@python.org wrote: > > Hey, Stefano! > > The level of being pedantic is absolutely acceptable. > > I don't question any of your arguments. They are all perfectly valid. > > Except that I'd rather say it is ~29 seconds if measuring against 1970. > Leap seconds were introduced in 1972 and there were > a total of 27 seconds since then, but TAI time was ticking since 1958 and > gained 10 seconds by 1970 so it is approximately 0.83 second per year at > which gives approx 28.67 sec between today and 1970. > So 1970 is a bad choice of epoch if you want to introduce a > leap-second-aware datetime. > In GPS time they chose 1980. In TAI it is 1958, but that is somewhat worse > than 1980 because it is not immediately clear how to perform the conversion > timestamp<->timedelta between 1958 and 1970. > > Something like 'proleptic gps time' would be needed to estimate the number > of leap seconds in the years before 1972 when they were introduced. Or > maybe to limit the leap-second timescale > to start at 1972 and not to accept any timestamps before that date. > > The system that ignores the existence of the leap seconds has a right to > exist. > But it just has limited applicability. > > np.datetime64 keeps time as a delta between the moment in time and a > predefined epoch. > Which standard does it use to translate this delta to human-readable time > in years, > months, and so on? > > If it is UTC, then it must handle times like 2016-12-31 23:59:60, because > it is a valid UTC timestamp. > >>> np.datetime64('2016-12-31 12:59:60') > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: Seconds out of range in datetime string "2016-12-31 12:59:60" > > Datetime also fails (so far) to handle it: > >>> dt(2016,12,31,23,59,60) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: second must be in 0..59 > > But `time` works. Well, at least it doesn't raise an exception: > >>> t = time.struct_time((2016,12,31,12,59,60,0,0,0)); t > time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=12, > tm_min=59, tm_sec=60, tm_wday=0, tm_yday=0, tm_isdst=0) > >>> time.asctime(t) > 'Mon Dec 31 12:59:60 2016' > >>> time.gmtime(calendar.timegm(t)) > time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, > tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) > > Imagine a user that decides which library to use to store some (life > critical!) measurements taken every 100 ms. He looks at NumPy datetime64, > reads that it is capable of handling attosecods, and decides that it is a > perfect fit. Now imagine that on 31 Dec 2022 the World Government decided > to inject a leap second. The system will receive the announcement from the > NTC servers and will > prepare to replay this second twice. As soon as this moment chimes in > he'll run into a ValueError, which he won't notice because he's celebrating > a New Year :) And guess whom he'll blame? ;) > > Actually the humanity has already got used to replaying timespans twice. > It happens every year in the countries that observe daylight saving time. > And the solution is to use a more linear scale than local time, namely, > UTC. But now turns out that UTC is not linear enough and it also has > certain timespans happening twice. > > The solution once again is use a _really_ linear time which is TAI. I > think python 'time' library did a right thing to introduce time.CLOCK_TAI, > after all. > > Astropy handles the UTC scale properly though: > >>> t = Time('2016-12-31 23:59:60') > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> > >>> t0 = Time('2016-12-31 23:59:59') > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:59.000> > >>> delta = t-t0 > <TimeDelta object: scale='tai' format='jd' value=1.1574074074038876e-05> > >>> delta.sec > 0.9999999999969589 > >>> t0 + delta > <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> > > So the solution for that particular person with regular intervals of time > is to use astropy. I mention it in the article. > I made some corrections to the text. I'd be grateful if you had a look and > pointed me to the particular sentences > that need improvement. > > Best regards, > Lev > > > _______________________________________________ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: lev.maxi...@gmail.com >
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
