[issue15443] datetime module has no support for nanoseconds

Gareth Rees added the comment: I also have a use case that would benefit from nanosecond resolution in Python's datetime objects, that is, representing and querying the results of clock_gettime() in a program trace. On modern Linuxes with a vDSO, clock_gettime() does not require a system

[issue15443] datetime module has no support for nanoseconds

Mark Dickinson added the comment: [Alexander] > Is there high enough demand for nanoseconds in datetime and time instances? One need that we've encountered in real code is simply for compatibility. We have Python code that interacts with a logging web service whose timestamps include

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: In the confines of PTP / IEEE1588, it's actually quite common and can be useful. It's not so much the ns, but the <1us that is missing. -- ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: > In telemetry, a nanosecond often translates to about a foot and 5 hours gets you to Pluto. Telemetry is exactly an application where absolute timestamps rarely make any sense. -- ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: This brings me back some times. Sorry if I am not up to date, the issue as I recall from back then was there wasn't even microseconds. In telemetry, you can often have these kind time stamped measurements, it's not insignificant noise nobody cares about. --

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Is there high enough demand for nanoseconds in datetime and time instances? How often nanosecond timestamps contain anything other than 0s or garbage in the last three digits? In my experience, all people want to do with such timestamps is to convert

[issue15443] datetime module has no support for nanoseconds

Paul Ganssle added the comment: > I don't think full nanosecond support is feasible to complete in the > remaining weeks This may be so, but I think the important part of that question is "what work needs to be done and what questions need to be answered?" If the answer is that we need to

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: @pganssle - let's keep the substantive discussions in the tracker so that they are not lost on github. You wrote: """ what is still blocking / needs to be done on this? Beta freeze for Python 3.10 is coming up at the beginning of May and I think we

[issue15443] datetime module has no support for nanoseconds

Change by Roundup Robot : -- keywords: +patch nosy: +python-dev nosy_count: 20.0 -> 21.0 pull_requests: +21096 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/21987 ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

Change by Steve Holden : -- nosy: -holdenweb ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15443] datetime module has no support for nanoseconds

Change by Paul Ganssle : -- nosy: +p-ganssle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15443] datetime module has no support for nanoseconds

Change by Mark Dickinson : -- nosy: +mark.dickinson ___ Python tracker ___ ___

[issue15443] datetime module has no support for nanoseconds

Change by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___

[issue15443] datetime module has no support for nanoseconds

Change by Shlomo Anglister : -- nosy: +anglister ___ Python tracker ___ ___

[issue15443] datetime module has no support for nanoseconds

Change by Eli_B : -- nosy: +Eli_B ___ Python tracker ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Another advantage of a single nanoseconds field is that currently microseconds are packed in 3 bytes and nanoseconds would fit in 4 - a 1 byte increase, but to add a 0-999 field, one would need at least 2 bytes. -- versions: +Python 3.7 -Python

[issue15443] datetime module has no support for nanoseconds

Steve Holden added the comment: I agree on reflection that a single nanoseconds integral value makes more sense. This then requires refactoring of the existing code so that existing tests continue to pass using a microsecond property. Code using ONLY nanoseconds is a disjoint case, for which

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: I've seen a similar glitch. Reloading the page usually fixes the problem. > On Jul 20, 2016, at 11:37 AM, Steve Holden wrote: > > > Steve Holden added the comment: > > BTW, I presume it's a bug in the issue tracker that my

[issue15443] datetime module has no support for nanoseconds

Changes by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

R. David Murray added the comment: I doubt it is a bug in the tracker. I've seen that kind of thing when I am having network issues...the browser renders what it gets, and if it doesn't get it all it looks like the page ends early. -- ___ Python

[issue15443] datetime module has no support for nanoseconds

Tim Peters added the comment: FYI, I'm seeing the same kind of odd truncation Steve sees - but it goes away if I refresh the page. -- ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

Steve Holden added the comment: BTW, I presume it's a bug in the issue tracker that my view of this message ends after a few lines of msg166386? Makes it rather difficult to track the issue! -- ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Note that the patches attached so far to this issue are nowhere close to a complete implementation. I don't think a python-only prototype (a patch to datetime.py) would be hard to implement, but implementation would be easier if nanoseconds replaced

[issue15443] datetime module has no support for nanoseconds

Changes by Serhiy Storchaka : -- Removed message: http://bugs.python.org/msg270265 ___ Python tracker ___

[issue15443] datetime module has no support for nanoseconds

Steve Holden added the comment: Just wanted to add a couple of comments here in case there's any interest. In our missions to make the world's market data available we deal with financial exchanges, many of whom are already recording event data at nanosecond resolution. Further, I believe

[issue15443] datetime module has no support for nanoseconds

Steve Holden added the comment: Just wanted to add a couple of comments here in case there's any interest. In our missions to make the world's market data available we deal with financial exchanges, many of whom are already recording event data at nanosecond resolution. Further, I believe

[issue15443] datetime module has no support for nanoseconds

Changes by Tomi Kyöstilä tomi.kyost...@gmail.com: -- nosy: +tomikyos ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

Changes by Ethan Furman et...@stoneleaf.us: -- nosy: -ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Python 2 line is closed for new features, but you can start with the main line Lib/datetime.py which will probably work with python 2.7.9 after some minor tweaks. You should be able to publish the result on PyPI. Note that many new in 3.x modules are

[issue15443] datetime module has no support for nanoseconds

Steve added the comment: Although I don't know what I am doing (patching python), if someone could point me to the relevant files in 2.7.9 that need to be patched, I'm willing to see if I can do it. -- ___ Python tracker rep...@bugs.python.org

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: Alexander, The initial patch is indeed minimal. If it gains momentum and some level of acceptation, I'd be happy to do more amends and fixes as needed and recommended. As for 2.7.9 - I'm not sure it makes much sense going PyPI patch if it's not going to happen on 3.x?

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: I have no doubt this will get into 3.x once we have a working patch and backward compatibility issues are addressed. Given the amount of effort Victor has recently put in #22117 to rework CPython internal time handling to support nanoseconds, it will

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Matthieu, I don't see you adding nanoseconds to timedelta in your patch. Doesn't this mean that you would loose nanoseconds when you subtract one datetime from another? To anyone who wants to contribute to this effort, I would recommend starting with

[issue15443] datetime module has no support for nanoseconds

R. David Murray added the comment: Unfortunately no, that would be a new feature and so can't go into 2.7. Maybe someone could backport the work that has been done in this area so people could patch locally, but I don't think it is a small job and I'm pretty sure no one on the core team is

[issue15443] datetime module has no support for nanoseconds

Steve added the comment: Hi, This issue is causing my organization problems. We are using python 2.7.9 with pyodbc 3.0.7 The application DB is SQL Server and they have started using Datetime2 (see: https://msdn.microsoft.com/en-us/library/bb677335.aspx?f=255MSPPError=-2147217396) They did

[issue15443] datetime module has no support for nanoseconds

Changes by Anand B Pillai abpil...@gmail.com: -- nosy: +pythonhacker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: Intention of the patch was to keep it simple and limited to nanoseconds (per the report). Throwing in Decimal would work (and possibly bring further precision) but consider: datetime.fromnanoseconds(ns) vs datetime.fromtimestamp(Decimal(ts)) I find the former

[issue15443] datetime module has no support for nanoseconds

Serhiy Storchaka added the comment: Pickling is not backward compatible. I.e. older versions of Python couldn't unpickle datetime pickled in new Python. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: backward compatibility is implemented as 'new python can load old pickle'. isn't it what backward compatible means? The payload changed from 10 to 12 bytes to accomodate the nanoseconds, I don't know how to handle reverse-backward compatibility or if it's really

[issue15443] datetime module has no support for nanoseconds

Serhiy Storchaka added the comment: May be instead of adding new method datetime.fromnanoseconds() make datetime.fromtimestamp() to support Decimal (and other high-precision numerical types)? -- nosy: +serhiy.storchaka ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: patch in attachment is an attempt to provide the datetime type nanosecond support, handles pickle versioning, expose a new class method datetime.fromnanoseconds -- keywords: +patch nosy: +mdcb...@gmail.com Added file:

[issue15443] datetime module has no support for nanoseconds

Changes by mdcb mdcb...@gmail.com: ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15443] datetime module has no support for nanoseconds

mdcb added the comment: minor bug fixes and improvements in new attachment. -- Added file: http://bugs.python.org/file37512/datetime.nanosecond.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443

[issue15443] datetime module has no support for nanoseconds

Changes by Ethan Furman et...@stoneleaf.us: -- nosy: +ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___ Python-bugs-list

[issue15443] datetime module has no support for nanoseconds

STINNER Victor added the comment: My patch for the issue #22043 adds nanosecond precision to get the system clock. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___

[issue15443] datetime module has no support for nanoseconds

Antoine Pitrou added the comment: Le 14/07/2014 22:53, Tim Peters a écrit : That consumes exactly 10 bytes today. Add nanoseconds, and it will take at least 11 (if 4 bits are insanely squashed into the bytes currently devoted to microseconds), and more likely 12 (if nanoseconds are sanely

[issue15443] datetime module has no support for nanoseconds

Niklas Claesson added the comment: I would like to add a use case. Control systems for particle accelerators. We have ns, sometimes ps precision on timestamped data acquisitions and we would like to use Python to do calculations. -- nosy: +Niklas.Claesson

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Given that struct timespec defined as struct timespec { time_t tv_sec;/* seconds */ long tv_nsec; /* nanoseconds */ }; is slowly becoming the prevailing standard to represent time

[issue15443] datetime module has no support for nanoseconds

Antoine Pitrou added the comment: For the record, numpy's datetime and timedelta types have theoretical support for attoseconds. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: numpy's datetime64 and timedelta64 types are so utterly broken that I would only recommend studying them as a negative example of how not to design a date-time library. -- ___ Python tracker

[issue15443] datetime module has no support for nanoseconds

Tim Peters added the comment: A note from Guido, from about 2 years ago: https://mail.python.org/pipermail/python-dev/2012-July/121127.html TBH, I think that adding nanosecond precision to the datetime type is not unthinkable. You'll have to come up with some clever backward compatibility in

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: (there's no room for an extra 10 bits in the carefully arranged 8-byte internal representation) According to a comment on top of Include/datetime.h, the internal representation of datetime is 10, not 8 bytes. /* Fields are packed into successive

[issue15443] datetime module has no support for nanoseconds

Antoine Pitrou added the comment: Le 14/07/2014 21:37, Alexander Belopolsky a écrit : AFAIK, Python objects are allocated with at least 32-bit alignment, 64 bits, actually, when using obmalloc.c. -- ___ Python tracker rep...@bugs.python.org

[issue15443] datetime module has no support for nanoseconds

Tim Peters added the comment: Yup, it's definitely more than 8 bytes. In addition to the comments you quoted, an in-memory datetime object also has a full Python object header, a member to cache the hash code, and a byte devoted to saying whether or not a tzinfo member is present. Guessing

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: Guessing Guido was actually thinking about the pickle size No, pickle also comes with an overhead from datetime import * import pickle t = datetime.now() len(pickle.dumps(t)) 70 For the present discussion, DATETIME_DATASIZE is the only relevant

[issue15443] datetime module has no support for nanoseconds

Tim Peters added the comment: Of course pickles come with overheads too - don't be tedious ;-) The point is that the guts of the datetime pickling is this: basestate = PyBytes_FromStringAndSize((char *)self-data, _PyDateTime_DATETIME_DATASIZE); That

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: The following code demonstrates that we can pack year through second fields in 40 bits: #include stdio.h struct dt { unsigned year :14; /* 1- of 0-16,383 */ unsigned month :4; /* 1-12 of 0-16 */ unsigned day :5; /* 1-31 of 0-31 */

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: I suppose another possibility is to get rid of microseconds internally, and work with a single 4-byte nanosecond member. Yes, that is what I think we should do. I would also split it from the packed fields to give it a 32-bit alignment which should

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky added the comment: If alignment is not required (as is the case in pickle), we can pack year through second + nanosecond fields in 9 bytes. For backwards compatibility we should continue accepting 10-byte pickles, but we can write in a new 9-byte format. --

[issue15443] datetime module has no support for nanoseconds

Andrew Clegg added the comment: I would like to add a real-world use case I have for nanosecond-precision support. I deal with data loggers that are controlled by GPS clocks, and I am writing some processing software in Python that requires the input of high-precision timestamps for

[issue15443] datetime module has no support for nanoseconds

Changes by Ramchandra Apte maniandra...@gmail.com: -- nosy: +ramchandra.apte ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___

[issue15443] datetime module has no support for nanoseconds

STINNER Victor victor.stin...@gmail.com added the comment: See the PEP 410. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___

[issue15443] datetime module has no support for nanoseconds

Marc-Andre Lemburg m...@egenix.com added the comment: Vincenzo Ampolo wrote: Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment: This is a real use case I'm working with that needs nanosecond precision and lead me in submitting this request: most OSes let users capture network

[issue15443] datetime module has no support for nanoseconds

Alexander Belopolsky alexander.belopol...@gmail.com added the comment: On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: ... full C double precision for the time part of a timestamp, which covers nanoseconds just fine. No, it does not: import time t =

[issue15443] datetime module has no support for nanoseconds

Marc-Andre Lemburg m...@egenix.com added the comment: Marc-Andre Lemburg wrote: Alexander Belopolsky alexander.belopol...@gmail.com added the comment: On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: ... full C double precision for the time part of a

[issue15443] datetime module has no support for nanoseconds

Marc-Andre Lemburg m...@egenix.com added the comment: Alexander Belopolsky wrote: Alexander Belopolsky alexander.belopol...@gmail.com added the comment: On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: ... full C double precision for the time part of a

[issue15443] datetime module has no support for nanoseconds

Marc-Andre Lemburg m...@egenix.com added the comment: [Roundup's email interface again...] x = 86400.0 x == x + 1e-9 False x == x + 1e-10 False x == x + 1e-11 False x == x + 1e-12 True -- ___ Python tracker rep...@bugs.python.org

[issue15443] datetime module has no support for nanoseconds

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment: Have a look to this python dev mailing list thread too: http://mail.python.org/pipermail/python-dev/2012-July/121123.html -- ___ Python tracker rep...@bugs.python.org

[issue15443] datetime module has no support for nanoseconds

New submission from Vincenzo Ampolo vincenzo.amp...@gmail.com: As long as computers evolve time management becomes more precise and more granular. Unfortunately the standard datetime module is not able to deal with nanoseconds even if OSes are able to. For example if i do: print %.9f %

[issue15443] datetime module has no support for nanoseconds

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +belopolsky, haypo, lemburg versions: +Python 3.4 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___

[issue15443] datetime module has no support for nanoseconds

Changes by Vincenzo Ampolo vincenzo.amp...@gmail.com: -- components: +Library (Lib) -ctypes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___

[issue15443] datetime module has no support for nanoseconds

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___

[issue15443] datetime module has no support for nanoseconds

Marc-Andre Lemburg m...@egenix.com added the comment: Vincenzo Ampolo wrote: As long as computers evolve time management becomes more precise and more granular. Unfortunately the standard datetime module is not able to deal with nanoseconds even if OSes are able to. For example if i do:

[issue15443] datetime module has no support for nanoseconds

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment: On 07/24/2012 01:28 PM, Marc-Andre Lemburg wrote: I would be interested in an actual use case for this. Alice has a dataset with nanosecond granularity. He wants to make a python library to let Bob access the dataset. Nowadays Alice

[issue15443] datetime module has no support for nanoseconds

R. David Murray rdmur...@bitdance.com added the comment: I believe Marc-Andre was looking for an actual real-world use case rather than a hypothetical one. We discussed this briefly on the irc channel and we think Guido vetoed it on a YAGNI basis (we haven't checked the archives though...) so

[issue15443] datetime module has no support for nanoseconds

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment: This is a real use case I'm working with that needs nanosecond precision and lead me in submitting this request: most OSes let users capture network packets (using tools like tcpdump or wireshark) and store them using file formats

[issue15443] datetime module has no support for nanoseconds

Changes by Vincenzo Ampolo vincenzo.amp...@gmail.com: -- versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443 ___ ___

[issue15443] datetime module has no support for nanoseconds

R. David Murray rdmur...@bitdance.com added the comment: Are the nanosecond timestamps timestamps or strings? If they are timestamps it's not immediately obvious why you want to convert them to datetime objects, so motivating that would probably help. On the other hand the fact that you

[issue15443] datetime module has no support for nanoseconds

R. David Murray rdmur...@bitdance.com added the comment: Even if accepted this can't get fixed in 2.7, so removing that from versions. -- versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15443

[issue15443] datetime module has no support for nanoseconds

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment: On 07/24/2012 04:20 PM, R. David Murray wrote: R. David Murray rdmur...@bitdance.com added the comment: Are the nanosecond timestamps timestamps or strings? If they are timestamps it's not immediately obvious why you want to