[issue15443] datetime module has no support for nanoseconds

[Gareth Rees]

Gareth Rees  added the comment:

I also have a use case that would benefit from nanosecond resolution in 
Python's datetime objects, that is, representing and querying the results of 
clock_gettime() in a program trace.

On modern Linuxes with a vDSO, clock_gettime() does not require a system call 
and completes within a few nanoseconds. So Python's datetime objects do not 
have sufficient resolution to distinguish between adjacent calls to 
clock_gettime().

This means that, like Mark Dickinson above, I have to choose between using 
datetime for queries (which would be convenient) and accepting that nearby 
events in the trace may be indistinguishable, or implementing my own 
datetime-like data structure.

--
nosy: +g...@garethrees.org

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Mark Dickinson]

Mark Dickinson  added the comment:

[Alexander]

> Is there high enough demand for nanoseconds in datetime and time instances?

One need that we've encountered in real code is simply for compatibility. We 
have Python code that interacts with a logging web service whose timestamps 
include nanosecond information. Whether or not nanosecond resolution makes 
sense for those timestamps is a moot point: that's out of our control.

When representing information retrieved from that web service in Python-land, 
we have a problem. If datetime.datetime had nanosecond precision, then using 
datetime.datetime to represent the retrieved values would be a no-brainer. As 
it is, we face a choice between:

- truncating or rounding to microsecond precision, and losing information as a 
result (which is particularly problematic if we want to write records back at 
any point)
- representing in some indirect form (as a str, an integer number of 
nanoseconds, a (datetime, nanoseconds) tuple, ...) and requiring the user to 
convert values for plotting or other analysis
- writing our own non-standard ns-supporting datetime class, or looking for a 
3rd party library with that support

None of those choices are terrible, but none of them are particularly palatable 
compared with using a standard library solution. (FWIW, we went with option 2, 
returning nanoseconds since the Unix epoch as an int.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb  added the comment:

In the confines of PTP / IEEE1588, it's actually quite common and can be 
useful. It's not so much the ns, but the <1us that is missing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

> In telemetry,

a nanosecond often translates to about a foot and 5 hours gets you to Pluto.  
Telemetry is exactly an application where absolute timestamps rarely make any 
sense.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb  added the comment:

This brings me back some times. Sorry if I am not up to date, the issue as I 
recall from back then was there wasn't even microseconds.
In telemetry, you can often have these kind time stamped measurements, it's not 
insignificant noise nobody cares about.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Is there high enough demand for nanoseconds in datetime and time instances?

How often nanosecond timestamps contain anything other than 0s or garbage in 
the last three digits?

In my experience, all people want to do with such timestamps is to convert them 
to something expressed in hours, minutes and seconds rather than just a huge 
number of seconds and back without loosing the value.

A timedelta is almost always a decent replacement for either datetime or time 
in those cases and sometimes it is even preferable because arithmetically it is 
closer to numbers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Paul Ganssle]

Paul Ganssle  added the comment:

> I don't think full nanosecond support is feasible to complete in the 
> remaining weeks

This may be so, but I think the important part of that question is "what work 
needs to be done and what questions need to be answered?" If the answer is that 
we need to make 3 decisions and do the C implementation, that seems feasible to 
do in under a month. If the answer is that we've got 10 contentious UI issues 
and we probably want to go through the PEP process, I agree with your 
assessment of the timing. Regardless, we'll need to know what work needs to be 
done before we do it...

> but we can try to add nanoseconds to timedelta only.  The mixed datetime + 
> timedelta ops will still truncate, but many time-related  operations will be 
> enabled. I would even argue that when nanoseconds precision is required, it 
> is more often intervals no longer than a few days and rarely a specific point 
> in time.

To be honest, I don't find this very compelling and I think it will only 
confuse people. I think most people use `timedelta` to represent something you 
add or subtract to a `datetime`. Having the `nanoseconds` part of it truncate 
seems like it would be frustrating and counter-intuitive. 

>From the use cases in this thread: 
 - ns-precision timestamps: https://bugs.python.org/issue15443#msg180125
 - ns-precision timestamps: https://bugs.python.org/issue15443#msg223039
 - Your suggestion that `datetime` should be able to support what `timespec` 
does: https://bugs.python.org/issue15443#msg223042
 - ns-precision timestamps: https://bugs.python.org/issue15443#msg270266

So I don't think there's high enough demand for nanosecond-timedelta on its own 
that we need to rush it out there before datetime gets it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

@pganssle - let's keep the substantive discussions in the tracker so that they 
are not lost on github.  You wrote:

"""
what is still blocking / needs to be done on this? Beta freeze for Python 3.10 
is coming up at the beginning of May and I think we may have enough time to get 
this in before then. Probably would have been better to get it into an alpha 
release, but if we miss beta freeze it'll get pushed to 3.11, and I do think 
that nanosecond support is a desirable feature for a lot of people.

It might be good for us to get an explicit "to-do" list of concerns to be 
addressed before this can be merged.
"""

I don't think full nanosecond support is feasible to complete in the remaining 
weeks, but we can try to add nanoseconds to timedelta only.  The mixed datetime 
+ timedelta ops will still truncate, but many time-related  operations will be 
enabled.

I would even argue that when nanoseconds precision is required, it is more 
often intervals no longer than a few days and rarely a specific point in time.

--
versions: +Python 3.10 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 20.0 -> 21.0
pull_requests: +21096
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21987

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve Holden]

Change by Steve Holden :


--
nosy:  -holdenweb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Paul Ganssle]

Change by Paul Ganssle :


--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Shlomo Anglister]

Change by Shlomo Anglister :


--
nosy: +anglister

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Eli_B]

Change by Eli_B :


--
nosy: +Eli_B

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Another advantage of a single nanoseconds field is that currently microseconds 
are packed in 3 bytes and nanoseconds would fit in 4 - a 1 byte increase, but 
to add a 0-999 field, one would need at least 2 bytes.

--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve Holden]

Steve Holden added the comment:

I agree on reflection that a single nanoseconds integral value makes more 
sense. This then requires refactoring of the existing code so that existing 
tests continue to pass using a microsecond property.

Code using ONLY nanoseconds is a disjoint case, for which new tests will be 
required. It clearly cannot be expected to be backwards compatible with 
pre-implementation versions.

Does it make sense to define behaviour for cases where the user attempts to MIX 
microseconds and nanoseconds? One validation I would suggest if so is that in 
the presence of a microseconds specification a constraint of 0 <= nanoseconds < 
1000 must be imposed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I've seen a similar glitch. Reloading the page usually fixes the problem. 

> On Jul 20, 2016, at 11:37 AM, Steve Holden  wrote:
> 
> 
> Steve Holden added the comment:
> 
> BTW, I presume it's a bug in the issue tracker that my view of this message 
> ends after a few lines of msg166386? Makes it rather difficult to track the 
> issue!
> 
> --
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[R. David Murray]

R. David Murray added the comment:

I doubt it is a bug in the tracker.  I've seen that kind of thing when I am 
having network issues...the browser renders what it gets, and if it doesn't get 
it all it looks like the page ends early.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Tim Peters]

Tim Peters added the comment:

FYI, I'm seeing the same kind of odd truncation Steve sees - but it goes away 
if I refresh the page.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve Holden]

Steve Holden added the comment:

BTW, I presume it's a bug in the issue tracker that my view of this message 
ends after a few lines of msg166386? Makes it rather difficult to track the 
issue!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Note that the patches attached so far to this issue are nowhere close to a 
complete implementation.  I don't think a python-only prototype (a patch to 
datetime.py) would be hard to implement, but implementation would be easier if 
nanoseconds replaced microseconds in datetime and timedelta objects with new 
microsecond(s) becoming a computed property.

--
keywords:  -patch
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
Removed message: http://bugs.python.org/msg270265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve Holden]

Steve Holden added the comment:

Just wanted to add a couple of comments here in case there's any interest. In 
our missions to make the world's market data available we deal with financial 
exchanges, many of whom are already recording event data at nanosecond 
resolution.

Further, I believe the decision to use a separate nanoseconds field to be 
essentially correct. While  it may well introduce some arithmetical complexity 
its value in backwards compatibility should be regarded as paramount. If I 
understand it correctly, the new nanosecond resolution times would continue to 
be correctly handled (module loss of nanosecond resolution) when handled as 
current microsecond date-times.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve Holden]

Steve Holden added the comment:

Just wanted to add a couple of comments here in case there's any interest. In 
our missions to make the world's market data available we deal with financial 
exchanges, many of whom are already recording event data at nanosecond 
resolution.

Further, I believe the decision to use a separate nanoseconds field to be 
essentially correct. While  it may well introduce some arithmetical complexity 
its value in backwards compatibility should be regarded as paramount. If I 
understand it correctly, the new nanosecond resolution times would continue to 
be correctly handled (module loss of nanosecond resolution) when handled as 
current microsecond date-times.

--
nosy: +holdenweb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Tomi Kyöstilä]

Changes by Tomi Kyöstilä tomi.kyost...@gmail.com:


--
nosy: +tomikyos

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Ethan Furman]

Changes by Ethan Furman et...@stoneleaf.us:


--
nosy:  -ethan.furman

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Python 2 line is closed for new features, but you can start with the main line 
Lib/datetime.py which will probably work with python 2.7.9 after some minor 
tweaks.  You should be able to publish the result on PyPI.  Note that many new 
in 3.x modules are provided for 2.x this way.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve]

Steve added the comment:

Although I don't know what I am doing (patching python), if someone could point 
me to the relevant files in 2.7.9 that need to be patched, I'm willing to see 
if I can do it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb added the comment:

Alexander,

The initial patch is indeed minimal. If it gains momentum and some level of 
acceptation, I'd be happy to do more amends and fixes as needed and recommended.

As for 2.7.9 - I'm not sure it makes much sense going PyPI patch if it's not 
going to happen on 3.x?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I have no doubt this will get into 3.x once we have a working patch and 
backward compatibility issues are addressed.  Given the amount of effort Victor 
has recently put in #22117 to rework CPython internal time handling to support 
nanoseconds, it will be odd not to support nanoseconds in datetime.

On the substance, in your patch you have chosen to add nanoseconds as a 
separate field instead of changing microseconds to nanoseconds.  I don't think 
this is the right approach.  See msg223082.  Once you get to implementing 
timedelta arithmetics, you will see that carrying two subsecond fields will 
unnecessarily complicate the code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Matthieu,

I don't see you adding nanoseconds to timedelta in your patch.  Doesn't this 
mean that you would loose nanoseconds when you subtract one datetime from 
another?

To anyone who wants to contribute to this effort, I would recommend starting 
with pure python implementation in Lib/datetime.py .

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[R. David Murray]

R. David Murray added the comment:

Unfortunately no, that would be a new feature and so can't go into 2.7.  Maybe 
someone could backport the work that has been done in this area so people could 
patch locally, but I don't think it is a small job and I'm pretty sure no one 
on the core team is interested.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Steve]

Steve added the comment:

Hi,
This issue is causing my organization problems. We are using python 2.7.9 with 
pyodbc 3.0.7
The application DB is SQL Server and they have started using Datetime2 (see: 
https://msdn.microsoft.com/en-us/library/bb677335.aspx?f=255MSPPError=-2147217396)

They did this to ensure that transactions timestamps are more unique, specially 
when data is bulk uploaded into the DB.

Datetime2 supports to seven places. Our application is now getting timestamps 
that are truncated to 6 places, making them useless when comparing a timestamp 
= to others in the db.

This is a real world issue and we really need a fix. We are not able to migrate 
at the moment to python 3 due to other constraints.
Any chance someone can take Matthieu's patch and retro fit it to 2.7.9 (if that 
makes sense)?

--
nosy: +scoobydoo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Anand B Pillai]

Changes by Anand B Pillai abpil...@gmail.com:


--
nosy: +pythonhacker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb added the comment:

Intention of the patch was to keep it simple and limited to nanoseconds (per 
the report).

Throwing in Decimal would work (and possibly bring further precision) but 
consider:

datetime.fromnanoseconds(ns)

vs

datetime.fromtimestamp(Decimal(ts))

I find the former cleaner - sure, it adds a new class method.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Pickling is not backward compatible. I.e. older versions of Python couldn't 
unpickle datetime pickled in new Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb added the comment:

backward compatibility is implemented as 'new python can load old pickle'. 
isn't it what backward compatible means?

The payload changed from 10 to 12 bytes to accomodate the nanoseconds, I don't 
know how to handle reverse-backward compatibility or if it's really needed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

May be instead of adding new method datetime.fromnanoseconds() make 
datetime.fromtimestamp() to support Decimal (and other high-precision numerical 
types)?

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb added the comment:

patch in attachment is an attempt to provide the datetime type nanosecond 
support, handles pickle versioning, expose a new class method 
datetime.fromnanoseconds

--
keywords: +patch
nosy: +mdcb...@gmail.com
Added file: http://bugs.python.org/file37509/datetime.nanosecond.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

Changes by mdcb mdcb...@gmail.com:


___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[mdcb]

mdcb added the comment:

minor bug fixes and improvements in new attachment.

--
Added file: http://bugs.python.org/file37512/datetime.nanosecond.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Ethan Furman]

Changes by Ethan Furman et...@stoneleaf.us:


--
nosy: +ethan.furman

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[STINNER Victor]

STINNER Victor added the comment:

My patch for the issue #22043 adds nanosecond precision to get the system clock.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le 14/07/2014 22:53, Tim Peters a écrit :

 That consumes exactly 10 bytes today. Add nanoseconds, and it will
take at least 11 (if 4 bits are insanely squashed into the bytes
currently devoted to microseconds), and more likely 12 (if nanoseconds
are sanely given their own 2 bytes).

That doesn't have to be. For example you could use the MSB of the 
microseconds field to store a datetime pickle flags byte, which could 
tell the unserializer whether a nanoseconds (or attoseconds :-)) field 
follows or not.

Remember that existing pickles must remain readable, so there must be 
some kind of version field anyway.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Niklas Claesson]

Niklas Claesson added the comment:

I would like to add a use case. Control systems for particle accelerators. We 
have ns, sometimes ps precision on timestamped data acquisitions and we would 
like to use Python to do calculations.

--
nosy: +Niklas.Claesson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Given that struct timespec defined as 

   struct timespec {
   time_t   tv_sec;/* seconds */
   long tv_nsec;   /* nanoseconds */
   };

is slowly becoming the prevailing standard to represent time in system 
interfaces, Python's inability to faithfully store it in a high level object 
will increasingly become a handicap.

People are starting to put nanoseconds in their databases not because they 
really need such precision, but because this is what they get from their 
devices and at the collection time cannot do anything smart.

The program that collects the events may simply not have time to do anything 
other than store raw data, or not have the higher level knowledge of what is 
the proper rounding.

The proper rounding is best to be done at the analysis time and by a program 
written in a higher level language such as Python.

--
assignee:  - belopolsky
stage:  - needs patch
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Antoine Pitrou]

Antoine Pitrou added the comment:

For the record, numpy's datetime and timedelta types have theoretical support 
for attoseconds.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

numpy's datetime64 and timedelta64 types are so utterly broken that I would 
only recommend studying them as a negative example of how not to design a 
date-time library.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Tim Peters]

Tim Peters added the comment:

A note from Guido, from about 2 years ago:

https://mail.python.org/pipermail/python-dev/2012-July/121127.html


TBH, I think that adding nanosecond precision to the datetime type is
not unthinkable. You'll have to come up with some clever backward
compatibility in the API though, and that will probably be a bit ugly
(you'd have a microsecond parameter with a range of 0-100 and a
nanosecond parameter with a range of 0-1000). Also the space it takes
in memory would probably increase (there's no room for an extra 10
bits in the carefully arranged 8-byte internal representation).


Add pickle, etc.

--
nosy: +tim.peters

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

 (there's no room for an extra 10 bits in the
 carefully arranged 8-byte internal representation)

According to a comment on top of Include/datetime.h, the internal 
representation of datetime is 10, not 8 bytes.


/* Fields are packed into successive bytes, each viewed as unsigned and
 * big-endian, unless otherwise noted:
 *
 * byte offset
 *  0   year 2 bytes, 1-
 *  2   month1 byte, 1-12
 *  3   day  1 byte, 1-31
 *  4   hour 1 byte, 0-23
 *  5   minute   1 byte, 0-59
 *  6   second   1 byte, 0-59
 *  7   usecond  3 bytes, 0-99
 * 10
 */

(if you don't trust the comments check the definitions a few lines below)

#define _PyDateTime_DATETIME_DATASIZE 10

AFAIK, Python objects are allocated with at least 32-bit alignment, so we have 
at least 2 unused bytes at the end of each datetime object.

Furthermore, out of 24 bits allocated for microseconds, only 20 are used, so 
nanoseconds can be accommodated by adding a single byte to DATETIME_DATASIZE.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le 14/07/2014 21:37, Alexander Belopolsky a écrit :

 AFAIK, Python objects are allocated with at least 32-bit alignment,

64 bits, actually, when using obmalloc.c.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Tim Peters]

Tim Peters added the comment:

Yup, it's definitely more than 8 bytes.  In addition to the comments you 
quoted, an in-memory datetime object also has a full Python object header, a 
member to cache the hash code, and a byte devoted to saying whether or not a 
tzinfo member is present.

Guessing Guido was actually thinking about the pickle size - but that's 10 
bytes (for a naive datetime object).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

 Guessing Guido was actually thinking about the pickle size

No, pickle also comes with an overhead

 from datetime import *
 import pickle
 t = datetime.now()
 len(pickle.dumps(t))
70

For the present discussion, DATETIME_DATASIZE is the only relevant number 
because we are not going to change anything other than the payload layout in 
the datetime object or its pickle serialization.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Tim Peters]

Tim Peters added the comment:

Of course pickles come with overheads too - don't be tedious ;-)  The point is 
that the guts of the datetime pickling is this:

basestate = PyBytes_FromStringAndSize((char *)self-data,
   _PyDateTime_DATETIME_DATASIZE);

That consumes exactly 10 bytes today.  Add nanoseconds, and it will take at 
least 11 (if 4 bits are insanely squashed into the bytes currently devoted to 
microseconds), and more likely 12 (if nanoseconds are sanely given their own 2 
bytes).  I suppose another possibility is to get rid of microseconds 
internally, and work with a single 4-byte nanosecond member.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

The following code demonstrates that we can pack year through second fields in 
40 bits:

#include stdio.h
struct dt {
  unsigned year :14;   /* 1- of 0-16,383 */
  unsigned month :4;   /* 1-12 of 0-16 */
  unsigned day   :5;   /* 1-31 of 0-31 */
  unsigned hour  :5;   /* 0-23 of 0-31 */
  unsigned minute:6;   /* 0-59 of 0-63 */
  unsigned second:6;   /* 0-59 of 0-63 */
  /* total : 40 bits */
};

int main() {
  struct dt t;
  t.year = ;
  t.month = 12;
  t.day = 31;
  t.hour = 24;
  t.minute = 59;
  t.second = 59;
  printf(%d-%d-%dT%d:%d:%d in %d bytes\n,
 t.year, t.month, t.day,
 t.hour, t.minute, t.second,
 (int)sizeof(t));
}

$ ./a.out
-12-31T24:59:59 in 8 bytes

Assuming 64-bit alignment, this leaves 64*2 - 40 = 88 bits for the sub-second 
field.

In 88 bits you can pack yoctoseconds (yocto = 1e-24) without breaking a sweat:

 2**88
309485009821345068724781056
 10**24
1

The practical choice is between 32-bit nanoseconds (nano = 1e-9) and 64-bit 
attoseconds (atto = 1e-18).

Given that the current the world record for shortest controllable time is 12 
attoseconds [1], it may not be an absurd choice to go to 64 bits of sub-second 
precision.

On the other hand, if we manage to go from micro- to nano-seconds now, I don't 
think it will be hard to go to higher resolution when users start asking for it 
some 40 years into the future.


[1] http://phys.org/news192909576.html

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

 I suppose another possibility is to get rid of microseconds
 internally, and work with a single 4-byte nanosecond member.

Yes, that is what I think we should do.  I would also split it from the packed 
fields to give it a 32-bit alignment which should improve performance.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

If alignment is not required (as is the case in pickle), we can pack year 
through second + nanosecond fields in 9 bytes.  For backwards compatibility we 
should continue accepting 10-byte pickles, but we can write in a new 9-byte 
format.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Andrew Clegg]

Andrew Clegg added the comment:

I would like to add a real-world use case I have for nanosecond-precision 
support. I deal with data loggers that are controlled by GPS clocks, and I am 
writing some processing software in Python that requires the input of 
high-precision timestamps for calculating clock drifts and offsets. The 
addition of nanosecond-precision support in datetime would allow me to use this 
rather than a homebrew solution.

--
nosy: +andrewclegg

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Ramchandra Apte]

Changes by Ramchandra Apte maniandra...@gmail.com:


--
nosy: +ramchandra.apte

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[STINNER Victor]

STINNER Victor victor.stin...@gmail.com added the comment:

See the PEP 410.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Marc-Andre Lemburg]

Marc-Andre Lemburg m...@egenix.com added the comment:

Vincenzo Ampolo wrote:
 
 Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment:
 
 This is a real use case I'm working with that needs nanosecond precision
 and lead me in submitting this request:
 
 most OSes let users capture network packets (using tools like tcpdump or
 wireshark) and store them using file formats like pcap or pcap-ng. These
 formats include a timestamp for each of the captured packets, and this
 timestamp usually has nanosecond precision. The reason is that on
 gigabit and 10 gigabit networks the frame rate is so high that
 microsecond precision is not enough to tell two frames apart.
 pcap (and now pcap-ng) are extremely popular file formats, with millions
 of files stored around the world. Support for nanoseconds in datetime
 would make it possible to properly parse these files inside python to
 compute precise statistics, for example network delays or round trip times.
 
 Other case is in stock markets. In that field information is timed in
 nanoseconds and have the ability to easily deal with this kind of
 representation natively with datetime can make the standard module even
 more powerful.
 
 The company I work for is in the data networking field, and we use
 python extensively. Currently we rely on custom code to process
 timestamps, a nanosecond datetime would let us avoit that and use
 standard python datetime module.

Thanks for the two use cases.

You might want to look at mxDateTime and use that for your timestamps.
It does provide full C double precision for the time part of a timestamp,
which covers nanoseconds just fine.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Alexander Belopolsky]

Alexander Belopolsky alexander.belopol...@gmail.com added the comment:

On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org 
wrote:
 ... full C double precision for the time part of a timestamp,
 which covers nanoseconds just fine.

No, it does not:

 import time
 t = time.time()
 t + 5e-9 == t
True

In fact, C double precision is barely enough to cover microseconds:

 t + 1e-6 == t
False

 t + 1e-7 == t
True

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Marc-Andre Lemburg]

Marc-Andre Lemburg m...@egenix.com added the comment:

Marc-Andre Lemburg wrote:
 
 Alexander Belopolsky alexander.belopol...@gmail.com added the comment:

 On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org 
 wrote:
 ... full C double precision for the time part of a timestamp,
 which covers nanoseconds just fine.

 No, it does not:

 import time
 t = time.time()
 t + 5e-9 == t
 True

 In fact, C double precision is barely enough to cover microseconds:

 t + 1e-6 == t
 False

 t + 1e-7 == t
 True
 
 I was referring to the use of a C double to store the time part
 in mxDateTime. mxDateTime uses the C double to store the number of
 seconds since midnight, so you don't run into the Unix ticks value
 range problem you showcased above.

There's enough room to even store 1/100th of a nanosecond, which may
be needed for some physics experiments :-)

False
 x == x + 1e-10
False
 x == x + 1e-11
False
 x == x + 1e-12
True

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Marc-Andre Lemburg]

Marc-Andre Lemburg m...@egenix.com added the comment:

Alexander Belopolsky wrote:
 
 Alexander Belopolsky alexander.belopol...@gmail.com added the comment:
 
 On Wed, Jul 25, 2012 at 4:17 AM, Marc-Andre Lemburg rep...@bugs.python.org 
 wrote:
 ... full C double precision for the time part of a timestamp,
 which covers nanoseconds just fine.
 
 No, it does not:
 
 import time
 t = time.time()
 t + 5e-9 == t
 True
 
 In fact, C double precision is barely enough to cover microseconds:
 
 t + 1e-6 == t
 False
 
 t + 1e-7 == t
 True

I was referring to the use of a C double to store the time part
in mxDateTime. mxDateTime uses the C double to store the number of
seconds since midnight, so you don't run into the Unix ticks value
range problem you showcased above.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Marc-Andre Lemburg]

Marc-Andre Lemburg m...@egenix.com added the comment:

[Roundup's email interface again...]

 x = 86400.0
 x == x + 1e-9
 False
 x == x + 1e-10
 False
 x == x + 1e-11
 False
 x == x + 1e-12
 True

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment:

Have a look to this python dev mailing list thread too:

http://mail.python.org/pipermail/python-dev/2012-July/121123.html

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

New submission from Vincenzo Ampolo vincenzo.amp...@gmail.com:

As long as computers evolve time management becomes more precise and more 
granular.
Unfortunately the standard datetime module is not able to deal with nanoseconds 
even if OSes are able to. For example if i do:

print %.9f % time.time()
1343158163.471209049

I've actual timestamp from the epoch with nanosecond granularity.

Thus support for nanoseconds in datetime would really be appreciated

--
components: ctypes
messages: 166326
nosy: Vincenzo.Ampolo
priority: normal
severity: normal
status: open
title: datetime module has no support for nanoseconds
type: enhancement
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +belopolsky, haypo, lemburg
versions: +Python 3.4 -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Changes by Vincenzo Ampolo vincenzo.amp...@gmail.com:


--
components: +Library (Lib) -ctypes

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Marc-Andre Lemburg]

Marc-Andre Lemburg m...@egenix.com added the comment:

Vincenzo Ampolo wrote:
 
 As long as computers evolve time management becomes more precise and more 
 granular.
 Unfortunately the standard datetime module is not able to deal with 
 nanoseconds even if OSes are able to. For example if i do:
 
 print %.9f % time.time()
 1343158163.471209049
 
 I've actual timestamp from the epoch with nanosecond granularity.
 
 Thus support for nanoseconds in datetime would really be appreciated

I would be interested in an actual use case for this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment:

On 07/24/2012 01:28 PM, Marc-Andre Lemburg wrote:
 I would be interested in an actual use case for this.

Alice has a dataset with nanosecond granularity. He wants to make a
python library to let Bob access the dataset. Nowadays Alice has to
implement her own time class losing all the flexibility of the datetime
module. With this enhancement she can provide a library that just uses
the standard python datetime module. Her library will get the needed
time format, including nanoseconds.

Many python sql libraries, like the one in django e the one in web2py,
relay on datetime objects for time representation. Bob has a web2py
website that has some data with nanosecond granularity. Nowadays Bob has
to store this data as a string or a long number without the ability to
use the powerful datetime module. With this enhancement Bob doesn't need
to build or learn another interface, he can just use the datetime module
using microseconds or nanoseconds as needed.

Google search for python datetime nanoseconds shows more than 141k
results:
https://www.google.com/search?sourceid=chromeie=UTF-8q=python+time#hl=enbiw=1615bih=938sclient=psy-abq=python+datetime+nanosecondsoq=python+datetime+nanoseconds

So this is definitively a requested feature. And as soon as technology
evolves more people will ask for it.

I imagine something like:

import datetime
nano_time = datetime.datetime(year=2012, month=07, day=24, hour=14,
minute=35, second=3, microsecond=53, nanosecond=27)

in case you need nanosecond granularity. if you don't need it just skip
the nanosecond part and the module works like it's now. Of course
strftime format should be updated to support nanoseconds.

I can write a patch if some dev can maybe review it.

Before someone takes the datetime source code and starts a third part
module that supports nanoseconds, I think this enhancement has almost
null impact in existing code and makes the datetime module even more
powerful. It's up to the Cpython admins to decide between maintaining
datetime module up to date with new needs or let third part modules take
care of those lacks.

Best Regards,
-- 
Vincenzo Ampolo
http://vincenzo-ampolo.net
http://goshawknest.wordpress.com

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

I believe Marc-Andre was looking for an actual real-world use case rather than 
a hypothetical one.  We discussed this briefly on the irc channel and we think 
Guido vetoed it on a YAGNI basis (we haven't checked the archives though...) so 
a real world use case is probably required.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment:

This is a real use case I'm working with that needs nanosecond precision
and lead me in submitting this request:

most OSes let users capture network packets (using tools like tcpdump or
wireshark) and store them using file formats like pcap or pcap-ng. These
formats include a timestamp for each of the captured packets, and this
timestamp usually has nanosecond precision. The reason is that on
gigabit and 10 gigabit networks the frame rate is so high that
microsecond precision is not enough to tell two frames apart.
pcap (and now pcap-ng) are extremely popular file formats, with millions
of files stored around the world. Support for nanoseconds in datetime
would make it possible to properly parse these files inside python to
compute precise statistics, for example network delays or round trip times.

Other case is in stock markets. In that field information is timed in
nanoseconds and have the ability to easily deal with this kind of
representation natively with datetime can make the standard module even
more powerful.

The company I work for is in the data networking field, and we use
python extensively. Currently we rely on custom code to process
timestamps, a nanosecond datetime would let us avoit that and use
standard python datetime module.

Best Regards,

---
Vincenzo Ampolo
http://vincenzo-ampolo.net
http://goshawknest.wordpress.com

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Changes by Vincenzo Ampolo vincenzo.amp...@gmail.com:


--
versions: +Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Are the nanosecond timestamps timestamps or strings?  If they are timestamps 
it's not immediately obvious why you want to convert them to datetime objects, 
so motivating that would probably help.  On the other hand the fact that you 
have an application that does so is certain an argument for real world 
applicability.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Even if accepted this can't get fixed in 2.7, so removing that from versions.

--
versions:  -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15443] datetime module has no support for nanoseconds

[Vincenzo Ampolo]

Vincenzo Ampolo vincenzo.amp...@gmail.com added the comment:

On 07/24/2012 04:20 PM, R. David Murray wrote:
 R. David Murray rdmur...@bitdance.com added the comment:
 
 Are the nanosecond timestamps timestamps or strings?  If they are timestamps 
 it's not immediately obvious why you want to convert them to datetime 
 objects, so motivating that would probably help.  On the other hand the fact 
 that you have an application that does so is certain an argument for real 
 world applicability.

It depends. When they are exported for example as csv (this can be the
case of market stock) or json (which is close to my case) that's a
string so having a datetime object may be very helpful in doing datetime
adds, subs, , deltas and in changing representation to human readable
format thanks to strftime() without loosing precison and maintaining
readability.

Think about a web application. User selects year, month, day, hour,
minute, millisecond, nanosecond of an event and the javascript does a
ajax call with time of this format (variant of iso8601):
-MM-DDTHH:MM:SS.mmnnn (where nnn is the nanosecond representation).
The python server takes that string, converts to a datetime, does all
the math with its data and gives the output back using labeling data
with int(nano_datetime.strftime('MMSSmmnnn')) so I've a sequence
number that javascript can sort and handle easily.

It's basically the same you already do nowadays at microseconds level,
but this time you have to deal with nanosecond data.

I agree with the YAGNI principle and I think that we have a clear
evidence of a real use case here indeed.

Best Regards

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15443
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com