[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
superseder: Add aware local time support to datetime module -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 6671c5039e15 by Alexander Belopolsky in branch 'default':
Issue #2736: Added datetime.timestamp() method.
http://hg.python.org/cpython/rev/6671c5039e15

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
components: +Documentation -Library (Lib)
resolution:  -> fixed
stage: test needed -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Roundup Robot]

Roundup Robot  added the comment:

New changeset b55eac85e39c by Alexander Belopolsky in branch 'default':
Issue #2736: Documented how to compute seconds since epoch.
http://hg.python.org/cpython/rev/b55eac85e39c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Thu, Apr 7, 2011 at 6:20 AM, Velko Ivanov  wrote:
..
>> Converting datetime values to float is easy.   If your dt is a naive 
>> instance representing UTC time:
>>
>>     timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
>>
>> If your dt is an aware instance:
>>
>>     timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / 
>> timedelta(seconds=1)
>
> Please add these lines to the datetime module's documentation. In some
> central, well lit place. I believe that if nothing else, the whole
> discussion should have proved to you that there are many people looking
> for them.

This is precisely what I suggested at the end of msg132697 above.  See
attached patch (issue2736-doc.diff) for a proposed documentation
enhancement.

--
Added file: http://bugs.python.org/file21565/issue2736-doc.diff

___
Python tracker 

___diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst
--- a/Doc/library/datetime.rst
+++ b/Doc/library/datetime.rst
@@ -720,6 +720,22 @@
out of the range of values supported by the platform C :c:func:`gmtime` 
function.
It's common for this to be restricted to years in 1970 through 2038. See 
also
:meth:`fromtimestamp`.
+
+   On the POSIX compliant platforms, ``utcfromtimestamp(timestamp)``
+   is equivalent to the following expression::
+
+ datetime(1970, 1, 1) + timedelta(seconds=timestamp)
+
+   There is no method to obtain the timestamp from a :class:`datetime`
+   instance, but POSIX timestamp corresponding to a :class:`datetime`
+   instance ``dt`` can be easily calculated as follows.  For a naive
+   ``dt``::
+
+  timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
+
+   And for an aware ``dt``::
+
+  timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / 
timedelta(seconds=1)
 
 
 .. classmethod:: datetime.fromordinal(ordinal)
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Velko Ivanov]

Velko Ivanov  added the comment:

On 04/05/2011 18:22, Alexander Belopolsky wrote:
> """
> The datetime module intended to be an island of relative sanity.
> ... """ - Tim Peters

Refusing to cooperate with the rest of the world is not sane by my books.

On 04/05/2011 21:06, Alexander Belopolsky wrote:
> Converting datetime values to float is easy.   If your dt is a naive instance 
> representing UTC time:
>
> timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
>
> If your dt is an aware instance:
>
> timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / 
> timedelta(seconds=1)

Please add these lines to the datetime module's documentation. In some 
central, well lit place. I believe that if nothing else, the whole 
discussion should have proved to you that there are many people looking 
for them.

OTOH a sinceepoch(epoch=datetime(1970,1,1)) method of the datetime class 
should be equally easy. Would be especially useful if few of the more 
frequently used EPOCHs are provided as constants.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Tue, Apr 5, 2011 at 1:45 PM, Marc-Andre Lemburg
 wrote:
..
> BTW: A "timestamp" usually refers to the combination of date and
> time. The time.time() return value is "seconds since the Epoch".
> I usually call those values "ticks" (not sure whether it's standard
> term of not, but always writing "seconds since Epoch" wasn't an
> option either ;-)).

In Unix context, the term "timestamp" is usually associated with the
various time values that OS stores with the files.  I think this use
is due to the analogy with physical "received on" timestamps used on
paper documents.  Since it is well-known that Unix filesystems store
time values as seconds since Epoch, it is common to refer to these
values as "Unix timestamps".

See, for example:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/touch.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Let me state my position on this issue once again.  Converting datetime values 
to float is easy.   If your dt is a naive instance representing UTC time:

   timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)

If your dt is an aware instance:

   timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / 
timedelta(seconds=1)

These recipes are easy to adjust for your application needs.  One application 
may want millisecond or microsecond ticks, another might want to carry 
subsecond presision in a separate integer, third may want to avoid timestamps 
before 1970 or after 2038 or ignore microseconds altogether.  No matter what a 
hypothetical datetime.epoch() will provide, most of applications will need to 
add a line or two to its code to serve their needs. Applications that will use 
dt.epoch() naively without thinking what dt represents (say local or UTC) will 
be buggy.

The only related feature that I think is missing from datetime module is the 
ability to obtain local time as an aware datetime instance and to convert a 
naive datetime instance assumed to represent local time to an aware one.

This is the subject of #9527, but there is a resistance to adding that feature.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Alexander Belopolsky wrote:
> 
> Alexander Belopolsky  added the comment:
> 
> MAL> Since most of the datetime module was inspired by mxDateTime,
> MAL> I wonder why [ticks()/gmticks()] were left out. (msg75411)
> 
> """
> The datetime module intended to be an island of relative sanity.
> Because the range of dates "timestamps" can represent varies across
> platforms (and even "the epoch" varies), datetime doesn't even try to
> produce timestamps directly -- datetime is more of an alternative to
> "seconds from the epoch" schemes. Because datetime objects have
> greater range and precision than timestamps, conversion is
> problem-free in only one direction. It's not a coincidence that
> that's the only direction datetime supplies ;-)
> """ - Tim Peters
>  
> http://bytes.com/topic/python/answers/522572-datetime-timestamp
> 
> I will also add that fromtimestamp() is not invertible in the presence of 
> DST.  That's why mxDatetime.ticks() takes a DST flag making it effectively a 
> multi-valued function.  Note that naive users, who just want to pass datetime 
> values to an OS function expecting a float, most likely will not have means 
> of properly obtaining DST flag.

IMHO, the whole concept of DST is broken, but that's not our fault :-)

Ditching the concept just because it is known to fail for
one hour out of 8760 you have in a typical year doesn't really
warrant breaking the "practicality beats purity" guideline.

Otherwise, we'd have to ditch the date support in the datetime module
too: after all, Feb 29 only exists every 4 years (well, most of the
time) - and that's one day out of 1461 in those 4 years, so an
even worse ratio :-)

And I'm not even starting to talk about ditching the concept
of Unix ticks to begin with, as a result of having leap seconds
causing POSIX ticks values not matching (real) UTC ticks.

In reality, all these things hardly ever matter and if they do,
users will either know that they have to make conscious decision,
simply don't care or decide not to care.

BTW: A "timestamp" usually refers to the combination of date and
time. The time.time() return value is "seconds since the Epoch".
I usually call those values "ticks" (not sure whether it's standard
term of not, but always writing "seconds since Epoch" wasn't an
option either ;-)).

Date/time is fun, isn't it ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Alexander Belopolsky wrote:
> 
> Alexander Belopolsky  added the comment:
> 
> On Tue, Apr 5, 2011 at 4:33 AM, Marc-Andre Lemburg
>  wrote:
> ..
>> mxDateTime, which in large parts inspired the Python datetime module,
>> has had a .ticks() method (for local time) and a .gmticks() method
>> (for UTC) for more than a decade now
> 
> Yes, mxDateTime's gmticks()/ticks() pair of functions present a much
> more mature design than anything that has been proposed here.  It is
> telling, however, that no one has mentioned mxDateTime's gmticks() on
> this issue in four years.  On a duplicate issue 1673409, Marc did
> bring it up, but as far as I can tell, no one responded.  See
> msg75411.
> 
> Google code search,
> 
> http://www.google.com/codesearch?hl=en&sa=N&q=gmticks+lang:python
> 
> returns only 13 hits for "gmticks".  In several instances, the
> resulting float is immediately converted to int, in other instances
> "gmticks" is mentioned in comments and the code works around its bugs.
> 
> I would not use Google Code search as an ultimate arbiter on the
> popularity of a feature, so I would really like to hear from the
> proponents about real life uses of gmticks() or any other examples
> where a similar method "has been reimplemented over and over again."

mxDateTime needs those two methods, since it doesn't natively
use timezones. The .ticks() method is used for local time values,
.gmticks() for UTC ones; that's why there are two methods.

The .gmticks() method is always used when storing UTC values
in mxDateTime instances, which actually is the preferred way
of storing data in databases. Google Code doesn't really count
much, since it only scans a limited number of OSS code bases.
Most of our users are commercial users who use the tools
in-house.

Note that implementing .gmticks() is fairly easy on POSIX
conform systems. On most others, timegm() can be used. If
that doesn't exist, things get tricky, but that case should
be rare nowadays.

>> so far, I haven't seen a single complaint about any of the issues raised in 
>> this discussion.
> 
> Well, search for gmticks does not return too many hits outside of
> mxDateTime code and manuals, but I had no trouble finding this:
> 
> """
>  okay, all the MySQLdb dataobject tick-based time handling methods are
>   broken in various ways -- reconstruct GMT ticks from time module's
>   mktime...
> """
> http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4251&dsMessageId=656863
> 
> Follow the link for some more colorful language describing developer's
> experience with the feature.
> 
> Note that it is likely that the bug MySQLdb developer complained about
> was fixed in mxDateTime at some point,
> , but
> this shows that implementing gmticks() correctly is not as trivial as
> those who never tried might think.

Note that he was referring to the .ticks() method, not the .gmticks()
method. The patch doesn't say which version of mxDateTime he was
using. The bug mentioned in the changelog was fixed in 1998. It is
possible, however, that the mktime() on his system was broken - which
is why I added a test for it in mxDateTime.

>>
>> The methods naturally return the Unix ticks value as float,
>> since that's what the time module uses as basis
> 
> Which in turn is a mistake IMO.  Note that POSIX does not use float
> timestamps for a reason.

The time module is our reference in this case and this tries hard
to add fractions of a second to the value :-)

Note that sub-second accuracy relies on a number of factors,
the storage format most certainly is the least important
aspect ;-)

On many systems, you only get 1/100s accuracy, on others, the timer
ticks in fixed increments, giving you even weirder sub-second values
(e.g. time appears to stay constant between time.time() calls).

OTOH, there's a new set of APIs for nano-second accuracy available
now, which the datetime module objects cannot represent at all due
to the integer-based storage format.

BTW: The integer format was chose in order to keep the memory
footprint of the objects low.

>> and the whole purpose
>> of those methods is to make interaction with the time module easy
>> and straight-forward.
> 
> This is not the goal that I would support.  I would rather see code
> that uses datetime module not require time module methods at all.

No chance :-) In practice, the time module gets used a lot
for date/time storage or to quickly measure time deltas.
Some people also prefer time module ticks due to their lower
memory footprint, esp. when it comes to storing thousands of
time values in time series.

>> Victor's patch could easily be updated to return floats as well,
>> to make it compatible with the time module.
>>
> 
> Victor reported implementing two methods, one to return a float and
> another to return a tuple. See msg124259.  I am not sure I've seen
> that code.

I had a look at the last patch on this ticket.

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

MAL> Since most of the datetime module was inspired by mxDateTime,
MAL> I wonder why [ticks()/gmticks()] were left out. (msg75411)

"""
The datetime module intended to be an island of relative sanity.
Because the range of dates "timestamps" can represent varies across
platforms (and even "the epoch" varies), datetime doesn't even try to
produce timestamps directly -- datetime is more of an alternative to
"seconds from the epoch" schemes. Because datetime objects have
greater range and precision than timestamps, conversion is
problem-free in only one direction. It's not a coincidence that
that's the only direction datetime supplies ;-)
""" - Tim Peters
 
http://bytes.com/topic/python/answers/522572-datetime-timestamp

I will also add that fromtimestamp() is not invertible in the presence of DST.  
That's why mxDatetime.ticks() takes a DST flag making it effectively a 
multi-valued function.  Note that naive users, who just want to pass datetime 
values to an OS function expecting a float, most likely will not have means of 
properly obtaining DST flag.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Tue, Apr 5, 2011 at 4:33 AM, Marc-Andre Lemburg
 wrote:
..
> mxDateTime, which in large parts inspired the Python datetime module,
> has had a .ticks() method (for local time) and a .gmticks() method
> (for UTC) for more than a decade now

Yes, mxDateTime's gmticks()/ticks() pair of functions present a much
more mature design than anything that has been proposed here.  It is
telling, however, that no one has mentioned mxDateTime's gmticks() on
this issue in four years.  On a duplicate issue 1673409, Marc did
bring it up, but as far as I can tell, no one responded.  See
msg75411.

Google code search,

http://www.google.com/codesearch?hl=en&sa=N&q=gmticks+lang:python

returns only 13 hits for "gmticks".  In several instances, the
resulting float is immediately converted to int, in other instances
"gmticks" is mentioned in comments and the code works around its bugs.

I would not use Google Code search as an ultimate arbiter on the
popularity of a feature, so I would really like to hear from the
proponents about real life uses of gmticks() or any other examples
where a similar method "has been reimplemented over and over again."

> so far, I haven't seen a single complaint about any of the issues raised in 
> this discussion.

Well, search for gmticks does not return too many hits outside of
mxDateTime code and manuals, but I had no trouble finding this:

"""
 okay, all the MySQLdb dataobject tick-based time handling methods are
  broken in various ways -- reconstruct GMT ticks from time module's
  mktime...
"""
http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4251&dsMessageId=656863

Follow the link for some more colorful language describing developer's
experience with the feature.

Note that it is likely that the bug MySQLdb developer complained about
was fixed in mxDateTime at some point,
, but
this shows that implementing gmticks() correctly is not as trivial as
those who never tried might think.

>
> The methods naturally return the Unix ticks value as float,
> since that's what the time module uses as basis

Which in turn is a mistake IMO.  Note that POSIX does not use float
timestamps for a reason.

> and the whole purpose
> of those methods is to make interaction with the time module easy
> and straight-forward.

This is not the goal that I would support.  I would rather see code
that uses datetime module not require time module methods at all.

> Victor's patch could easily be updated to return floats as well,
> to make it compatible with the time module.
>

Victor reported implementing two methods, one to return a float and
another to return a tuple. See msg124259.  I am not sure I've seen
that code.

> There's only one catch that Victor's patch doesn't include ...

No, it is not  about "only one catch". Victor's patch is simply wrong.
 For an aware datetime instance it extracts DST flag from tzinfo, but
ignores the offset.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
> Marc, could you maybe write a new patching taking care of the DST and
> maybe also the timezone? It looks like you have a long experience in
> timestamps :-)

Sorry, but no. I'm not really a fan of the datetime module and
try to stay away from it whenever I can :-)

Note that dealing with DST in timezones other than the local time
zone, is bound to go wrong without direct access to the tz library.
The C lib doesn't provide any good way to access timezone
information other than the local timezone or UTC.

When dealing with date/time values, it is usually best to stay with
UTC and only transform those values into local times in user
interfaces on the front-end client.

Consequently, I'd suggest to only allow UTC and local timezone
conversions for the method in the datetime module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[STINNER Victor]

STINNER Victor  added the comment:

Marc, could you maybe write a new patching taking care of the DST and
maybe also the timezone? It looks like you have a long experience in
timestamps :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Just to add another data point to this discussion:

mxDateTime, which in large parts inspired the Python datetime module,
has had a .ticks() method (for local time) and a .gmticks() method
(for UTC) for more than a decade now and so far, I haven't seen a
single complaint about any of the issues raised in this discussion.

The methods naturally return the Unix ticks value as float,
since that's what the time module uses as basis and the whole purpose
of those methods is to make interaction with the time module easy
and straight-forward. Likewise, the epoch is also the same as the time
module's one.

Victor's patch could easily be updated to return floats as well,
to make it compatible with the time module.

There's only one catch that Victor's patch doesn't include: mktime()
doesn't always work with DST set to anything but -1. mxDateTime
checks the API at module load time and then determines whether
it can be used with a DST setting or not (see the mxDateTime code
for details). Not sure whether today's mktime() implementations
still have any issues with this, but it's better to double-check
than to get wrong results.

http://www.egenix.com/products/python/mxBase/mxDateTime/

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Mon, Apr 4, 2011 at 5:42 PM, Jay Taylor  wrote:
..
> I couldn't agree more with ping's position on this.

Adding votes to a tracker issue without a working patch will not move
it any further.   There are several committers besides me in the nosy
list including the original author of the datetime module.  If it was
such a universally desired feature as Ka-Ping makes it sound, it would
be committed long before I became the maintainer of the datetime
module.

>  It is against the spirit of what Python has set out to be, and the blocking 
> needs to stop.

I don't think any committer has a power to *block* a patch.  I
certainly don't.  If  Ka-Ping wants to add a feature over my
objections, it is well within his power to do so. (Note that I
objected to timedelta.total_seconds(), but it was added nevertheless.)
 It would be best, however to bring this to python-dev or python-ideas
first.

> Any chance we could get a .epoch() function into python 2.7 as well?

No.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Jay Taylor]

Jay Taylor  added the comment:

I couldn't agree more with ping's position on this.  It is against the spirit 
of what Python has set out to be, and the blocking needs to stop.

Any chance we could get a .epoch() function into python 2.7 as well?

--
nosy: +Jay.Taylor

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Ka-Ping Yee]

Ka-Ping Yee  added the comment:

> no one has come up with a satisfactory solution

Plenty have proposed a satisfactory solution.  No one has come up with a 
solution that is satisfactory to *you*, because you have overconstrained the 
problem.  The reason we still have no utctotimestamp() after all these years is 
that you, and you alone as far as I know, refuse to accept a method that 
inverts utcfromtimestamp() with microsecond precision over its working range.  
Such a method is a perfectly reasonable and acceptable solution and would add a 
lot of value to Python as a language.

I suspect you don't realize just how much pain you have unintentionally caused 
the world of Python users by singlehandedly blocking progress on this issue.  
I've seen them: students, friends, coworkers -- even very smart and capable 
people are stymied by it.  No one thinks of looking in the calendar module.  
Maybe if you watched some of them struggle with this, you would understand.

> leave it as an exercise to the reader to solve

To take this perspective is to miss the point of Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Thu, Mar 31, 2011 at 2:52 PM, Ka-Ping Yee  wrote:
..
> I am extremely disappointed by what has happened here.
>

What exactly are you disappointed about?  As far as I can tell, the
feature request has not been rejected, just no one has come up with a
satisfactory solution.   The issue is open and patches are welcome.

> We are talking about a very simple method that everybody needs, and that has 
> been
> reimplemented over and over again.  I have been frustrated countless times by 
> the lack of
> a utctotimestamp() method.

This is not what this issue has been about so far.  It was about local
time to timestamp.  In py3k,  utctotimestamp() is easy:

EPOCH = datetime(1970, 1, 1)
def utctotimestamp(dt) :
  return (dt - EPOCH).total_seconds()

>  I have watched beginners and experienced programmers alike suffer over and 
> over
> again for the lack of this method, and spend hours trying to figure out why 
> Python
> doesn't have it and how it should be spelled in Python.
>

These "beginners and experienced programmers" may want to reconsider
using floating point numbers to store high precision timestamps.  I
know that some OSes made the same unfortunate choice in system APIs,
but it does not make this choice any better.   I can make a long list
of why this is a bad choice, but I'll just mention that the precision
of your timestamp varies from year to year and the program that works
fine today may mysteriously fail in 5 years when nobody is around who
can fix it anymore.

> The discussion here has been stuck on assumptions that the method must meet
> all of the following ideals:
>
>  1. It must produce a value that is easy to compute with
>  2. It must have perfect precision in representing microseconds, forever
>  3. It must make an exact round-trip for any possible input
>  4. It must let users use whatever epoch they want
>

No it was actually stuck because of the inability to reliably obtain
the system UTC offset for historical times.  This is a solvable
problem, but the patches proposed so far did not solve it correctly.
On top of this,  there is an issue of datetime.fromtimestamp() not
being invertible in the presence of DST shifts, so
datetime.totimestamp() is ambiguous for some datetime values.

> These ideals cannot all be met simultaneously and perfectly.  The correct 
> thing to
> do as an engineer is to choose a practical compromise and document the 
> decision.
>
> The compromise that almost everyone chooses (because it is useful, 
> convenient, has
> microsecond precision at least until the year 2100, and millisecond precision 
> is frequently
> sufficient) is to use a floating-point number with an epoch of 1970-01-01.  
> Floating-point
> seconds can be easily subtracted, added, serialized, and deserialized, and 
> are a primitive
> data type in nearly every language and database.

Those who need to do arithmetics on time values more often deal with
durations rather than points in time.   An arbitrary epoch around
current time is often more appropriate for timeseries analytics than
Unix epoch.

>  They are unmatched in ease of use.

Compared to what?  I find integers much more suitable for representing
points in time than floats.  Yes, in some languages you have to deal
with 32-bit int overflow issues if you want to be able to deal with
durations of over 100 years expressed in microseconds, but these days
64-bit integers are almost universally available.

>  So everyone wastes time searching for the answer and figuring out how to 
> write:
>
>    import calendar
>    calendar.timegm(dt.utctimetuple()) + dt.microsecond * 1e-6
>

And this is the wrong answer.  Someone else using (dt -
EPOCH).total_seconds() may get a slightly different result.  Some may
argue that given that it is not obvious what expression to use, we
need to provide a function.  However, we already provided
timedelta.total_seconds() that hides the floating point details.  In
my opinion, even adding total_seconds() was a mistake and x /
timedelta(seconds=1) is just as short and more explicit than
x.total_seconds().

I think the best we can do is to expand datetime.utcfromtimestamp()
documentation to explain that it is equivalent to

def utcfromtimestamp(s):
 return EPOCH + timedelta(seconds=s)

and either leave it as an exercise to the reader to solve
utcfromtimestamp(s) = dt for s or spell out

def utctotimestamp(dt) :
  return (dt - EPOCH) / timedelta(seconds=1)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Ka-Ping Yee]

Ka-Ping Yee  added the comment:

I am extremely disappointed by what has happened here.

We are talking about a very simple method that everybody needs, and that has 
been reimplemented over and over again.  I have been frustrated countless times 
by the lack of a utctotimestamp() method.  I have watched beginners and 
experienced programmers alike suffer over and over again for the lack of this 
method, and spend hours trying to figure out why Python doesn't have it and how 
it should be spelled in Python.

The discussion here has been stuck on assumptions that the method must meet all 
of the following ideals:

  1. It must produce a value that is easy to compute with
  2. It must have perfect precision in representing microseconds, forever
  3. It must make an exact round-trip for any possible input
  4. It must let users use whatever epoch they want

These ideals cannot all be met simultaneously and perfectly.  The correct thing 
to do as an engineer is to choose a practical compromise and document the 
decision.

The compromise that almost everyone chooses (because it is useful, convenient, 
has microsecond precision at least until the year 2100, and millisecond 
precision is frequently sufficient) is to use a floating-point number with an 
epoch of 1970-01-01.  Floating-point seconds can be easily subtracted, added, 
serialized, and deserialized, and are a primitive data type in nearly every 
language and database.  They are unmatched in ease of use.  So everyone wastes 
time searching for the answer and figuring out how to write:

import calendar
calendar.timegm(dt.utctimetuple()) + dt.microsecond * 1e-6

We should use this as the definition of datetime.utctotimestamp(), document its 
limitations, and be done with it.

Instead, this essential and useful method has now been held up for almost three 
YEARS by an inability to accept a simple engineering decision.  Unbelievable.

--
nosy: +ping

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[STINNER Victor]

STINNER Victor  added the comment:

It looks like it's not possible to choose between float and (int, int) output 
type for datetime.totimestamp(). One is more practical (and enough for people 
who doesn't need an exact result), and one is needed to keep the same 
resolution than the datetime object. I think that we can add two methods:
 * datetime.totimestamp()->float
 * datetime.totimestamptuple()->(int,int)

I choosed the shortest name for float because I suppose that most users prefer 
float than a tuple, and so the API is symmetrical:
 * datetime.fromtimestamp(float)->datetime
 * datetime.totimestamp()->float

My patch have to be updated to use the timezone (and the DST thing?) and also 
to update the Python implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, Dec 17, 2010 at 3:26 PM, Antoine Pitrou  wrote:
..
>
>> Yes, UTC not being a proper acronym in any human language is one
>> problem,
>
> Ok. Too bad you don't live on the same planet than most of us. I bail
> out.

Sorry that my attempt at humor has proven to be too subtle.  I was
referring to the following fact:

"""
The International Telecommunication Union wanted Coordinated Universal
Time to have the same symbol in all languages. English and French
speakers wanted the initials of both their respective language's terms
to be used internationally: "CUT" for "coordinated universal time" and
"TUC" for "temps universel coordonné". This resulted in the final
compromise of "UTC".
"""

http://en.wikipedia.org/wiki/Coordinated_Universal_Time

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Yes, UTC not being a proper acronym in any human language is one
> problem,

Ok. Too bad you don't live on the same planet than most of us. I bail
out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
nosy:  -Alexander.Belopolsky
versions: +Python 3.3 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, Dec 17, 2010 at 2:35 PM, Antoine Pitrou  wrote:
..
> I don't think the "time" module can be named "higher level", and it
> still handles such timestamps.
>
>> datetime(1970, 1, 1) + timedelta(seconds=s)
>>
>> is obvious, self-contained,  short and does not require any knowledge
>> other than elementary school arithmetic to understand.
>
> Sigh.
> Again: it's so obvious that you're the only one who seems to easily come
> up with those solutions. How many times does it have to be repeated?
>

Remember, most of the code is written once, but read and edited many
times.  Show me one person who will have trouble understanding what
datetime(1970, 1, 1) + timedelta(seconds=s) means and show me another
who can understand datetime.utcfromtimestamp(s) without reading the
manual.

>> Compared to
>> this, "utcfromtimestamp" is a monstrosity that suggests that something
>> non-trivial, such as UTC leap seconds is been taken care of.
>
> I don't see anything suggesting it is a monstrosity. The name is
> grammatically bizarre, but that's all.
>

Yes, UTC not being a proper acronym in any human language is one
problem, Python datetime not being able to represent some valid UTC
times is another.

That's correct, but most users expect their timestamps to be the same
when saved on one system and read on another.  Granted, most users
expect the same from their floats as well, but this can only be solved
by education.   Calendaric calculations are complex enough that we
don't want to expose users to floating point gotchas at the same time.

>> Note that when timedelta.total_seconds() was first committed, it
>> contained a numerical bug.  See issue8644.
>
> So? What is your point?
>

I thought the point was obvious: conversion between time values and
float is non-trivial and error prone.  Users should not be encouraged
to casually convert (seconds, microseconds) tuples to floats.  If they
do, chances are they will do it differently in different parts of the
program.

>> In any case, you ignore the hard question about totimestamp():
>> fromtimestamp() is not invertible in most real life timezones.  If you
>> have a solution that does not restrict totimestamp() to UTC, I would
>> like to hear it.
>
> IMO, the solution would have the datetime object carry the offset from
> UTC with it, rather than try to be smart and compute it dynamically.
>

Ditto.  This is exactly what issue9527 is attempting to achieve.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> > ??? EPOCH is not even a constant in the datetime module.
> >
> No, and it does not belong there.

And so what was your point exactly?

> A higher level library that uses
> seconds since epoch for interchange

I don't think the "time" module can be named "higher level", and it
still handles such timestamps.

> datetime(1970, 1, 1) + timedelta(seconds=s)
> 
> is obvious, self-contained,  short and does not require any knowledge
> other than elementary school arithmetic to understand.

Sigh.
Again: it's so obvious that you're the only one who seems to easily come
up with those solutions. How many times does it have to be repeated?

> Compared to
> this, "utcfromtimestamp" is a monstrosity that suggests that something
> non-trivial, such as UTC leap seconds is been taken care of.

I don't see anything suggesting it is a monstrosity. The name is
grammatically bizarre, but that's all.

> Let's see:
> [snip]
>
> which one is *obviously* correct?  Are they *obviously* equivalent?

Both are obviously correct for whatever the non-perverted user has in
mind. People in real life don't care whether they will retain
microsecond precision when carrying a floating point timestamp around.
For the simple reason that the data source itself will not have such
precision.

> Note that when timedelta.total_seconds() was first committed, it
> contained a numerical bug.  See issue8644.

So? What is your point?

> In any case, you ignore the hard question about totimestamp():
> fromtimestamp() is not invertible in most real life timezones.  If you
> have a solution that does not restrict totimestamp() to UTC, I would
> like to hear it.

IMO, the solution would have the datetime object carry the offset from
UTC with it, rather than try to be smart and compute it dynamically.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Miki Tebeka]

Changes by Miki Tebeka :


--
nosy:  -tebeka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, Dec 17, 2010 at 1:17 PM, Antoine Pitrou  wrote:
..
>> A better question is why datetime.utcfromtimestamp(s) exists given
>> that it is actually longer than equivalent EPOCH + timedelta(0, s)?
>
> ??? EPOCH is not even a constant in the datetime module.
>
No, and it does not belong there.  A higher level library that uses
seconds since epoch for interchange may define it (and make a decision
whether it should be a naive datetime(1970, 1, 1) or datetime(1970, 1,
1, tzinfo=timezone.utc)).

> And regardless, the point is *not* the number of characters typed, but
> how easy it is to come up with the solution. Calling the appropriate
> (and appropriately-named) method is much easier than coming up with the
> right datetime arithmetic incantation. It's Python, not Perl. "There
> should be one obvious way to do it".
>

I don't see anything obvious about the choice between
utcfromtimestamp(s), fromtimestamp(s) and utcfromtimestamp(s,
timezone.utc).

datetime(1970, 1, 1) + timedelta(seconds=s)

is obvious, self-contained,  short and does not require any knowledge
other than elementary school arithmetic to understand.  Compared to
this, "utcfromtimestamp" is a monstrosity that suggests that something
non-trivial, such as UTC leap seconds is been taken care of.

>> > And returning a (seconds, microseconds) tuple does retain the precision.
>> >
>>
>> It does, but it does not help much those who want a float - they would
>> still need another line of code.
>
> Yes, but a very obvious one at least.
>

Let's see:

def floattimestamp(t):
  s, us = t.totimestamp()
  return s + us * 1e-6

and

def floattimestamp(t):
  s, us = t.totimestamp()
  return s + us / 100

which one is *obviously* correct?  Are they *obviously* equivalent?

Note that when timedelta.total_seconds() was first committed, it
contained a numerical bug.  See issue8644.

>> Note that with divmod(timedelta,
>> timedelta), you can now easily extract   (seconds, microseconds)  or
>> any other tuple like (weeks, days, seconds. microseconds) from
>> timedelta objects.
>
> Do you think many users even think of calling divmod() timedelta
> objects? I don't, personally.
>
> You apparently hold the opinion that the datetime module should be
> reserved for experts in arithmetic over dates, times and timedeltas. But
> it's not. It's the Python stdlib and it should provide reasonably
> high-level tools to do the job.
>

Sure, but if the goal is to implement json serialization of datetime
objects, maybe stdlib should provide a high-level tool for *that* job?
  Using float representation of datetime is probably the worst option
for json: it is non-standard, may either loose information or
introduce spurious differences, and is not human-readable.

In any case, you ignore the hard question about totimestamp():
fromtimestamp() is not invertible in most real life timezones.  If you
have a solution that does not restrict totimestamp() to UTC, I would
like to hear it.  Otherwise, I don't see any problem with (t -
datetime(1970, 1, 1)).total_seconds() expression.  Maybe we can add
this recipe to utcfromtimestamp() documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> >> 1. Different application may need different epoch and retained
> >> precision depends on the choice of the epoch.
> >
> > But then why does fromtimestamp() exist?
> 
> A better question is why datetime.utcfromtimestamp(s) exists given
> that it is actually longer than equivalent EPOCH + timedelta(0, s)?

??? EPOCH is not even a constant in the datetime module.

And regardless, the point is *not* the number of characters typed, but
how easy it is to come up with the solution. Calling the appropriate
(and appropriately-named) method is much easier than coming up with the
right datetime arithmetic incantation. It's Python, not Perl. "There
should be one obvious way to do it".

> > And returning a (seconds, microseconds) tuple does retain the precision.
> >
> 
> It does, but it does not help much those who want a float - they would
> still need another line of code.

Yes, but a very obvious one at least.

> Note that with divmod(timedelta,
> timedelta), you can now easily extract   (seconds, microseconds)  or
> any other tuple like (weeks, days, seconds. microseconds) from
> timedelta objects.

Do you think many users even think of calling divmod() timedelta
objects? I don't, personally.

You apparently hold the opinion that the datetime module should be
reserved for experts in arithmetic over dates, times and timedeltas. But
it's not. It's the Python stdlib and it should provide reasonably
high-level tools to do the job.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, Dec 17, 2010 at 12:17 PM, Antoine Pitrou  wrote:
..
>> 1. Different application may need different epoch and retained
>> precision depends on the choice of the epoch.
>
> But then why does fromtimestamp() exist?

A better question is why datetime.utcfromtimestamp(s) exists given
that it is actually longer than equivalent EPOCH + timedelta(0, s)?  I
am not responsible for either of these methods, but at least
datetime.fromtimestamp(s, tz) is well defined for any timezone and
timestamp unlike its inverse.

> And returning a (seconds, microseconds) tuple does retain the precision.
>

It does, but it does not help much those who want a float - they would
still need another line of code.  Note that with divmod(timedelta,
timedelta), you can now easily extract   (seconds, microseconds)  or
any other tuple like (weeks, days, seconds. microseconds) from
timedelta objects.  See msg75904 above.

>> 2. The code above works only on naive datetime objects assumed to be
>> in UTC.
>
> So, if the "trivial" code doesn't work, you can't bring it up as an
> argument against shipping this functionality, right?
>

Well, no one has come up with the code that does work so far.  Note
that timetuple path does not work either because it does not fill
tm_isdst correctly.  The only solution I can think of for having
proper inverse to fromtimestamp() is to add isdst to datetime objects.
 This would allow correct round-tripping between datetime and
timetuple and datetime and timestamp.

>> 3. While it is not hard to extend the timestamp(t) code to cover aware
>> datetime objects that use fixed offset tzinfo such as those with
>> tzinfo set to a datetime.timezone instance, it is not well defined for
>> the "smart" tzinfo implementations that do automatic DST adjustment.
>
> Still, fromtimestamp() exists and apparently fulfills people's
> expectations. So why can't the same strategy be used for totimestamp()
> as well?

Because in certain timezones fromtimestamp() can return the same
datetime value for different timestamps and some datetime values do
not have a corresponding timestamp.  I have not seen a working
proposal on how to handle these issues yet.  You are asking to provide
an inverse to an existing function simply because the function exists.
 But the function in question is not invertible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> 1. Different application may need different epoch and retained
> precision depends on the choice of the epoch.

But then why does fromtimestamp() exist?
And returning a (seconds, microseconds) tuple does retain the precision.

> 2. The code above works only on naive datetime objects assumed to be
> in UTC.

So, if the "trivial" code doesn't work, you can't bring it up as an
argument against shipping this functionality, right?

> 3. While it is not hard to extend the timestamp(t) code to cover aware
> datetime objects that use fixed offset tzinfo such as those with
> tzinfo set to a datetime.timezone instance, it is not well defined for
> the "smart" tzinfo implementations that do automatic DST adjustment.

Still, fromtimestamp() exists and apparently fulfills people's
expectations. So why can't the same strategy be used for totimestamp()
as well?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, Dec 17, 2010 at 9:18 AM, R. David Murray  wrote:
>
> R. David Murray  added the comment:
>
> Alexander, I agree with Velko in that it isn't obvious to me how the addition 
> of localtime
> would answer the desire expressed in this issue.

Conversion of UTC datetime to time stamp is trivial:

EPOCH = datetime(1970, 1, 1)
def timestamp(t):
  return (t - EPOCH).total_seconds()

There are several reasons not to include this one-liner in stdlib
(other than it being a one-liner).

1. Different application may need different epoch and retained
precision depends on the choice of the epoch.

2. The code above works only on naive datetime objects assumed to be
in UTC.  Passing say a result of datetime.now() to it is likely to
result in a hard to find bug.

3. While it is not hard to extend the timestamp(t) code to cover aware
datetime objects that use fixed offset tzinfo such as those with
tzinfo set to a datetime.timezone instance, it is not well defined for
the "smart" tzinfo implementations that do automatic DST adjustment.
This is where the localtime (#9527) issue comes into play.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Velko Ivanov]

Velko Ivanov  added the comment:

> on the other hand, given Victor's research, I don't see float seconds since 
> an epoch appearing anywhere as a standard.  Where do you see this being used 
> as a standard?

Yes, I didn't mean standard as in RFCed and recommended and dominant, sorry if 
it sounded that way. I meant just that it is quite common in many places, big 
and small.

> I also don't understand your complaint about the fact that the one-liner 
> creates a timetuple.  datetime stores the date and time information as 
> discrete fields, so generating a timetuple is a natural conversion path.  

Well, the timetuple is not a tuple, but an object filled with attributes. It 
contains a few more than are required for this conversion and it doesn't 
contain one that is required. Therefore I really see that as an inelegant and 
ineffective way to do the conversion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Velko: on the other hand, given Victor's research, I don't see float
> seconds since an epoch appearing anywhere as a standard.

Well, given that we already have fromtimestamp(), this sounds like a
poor argument against a totimestamp() method (or whatever it gets
called).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[R. David Murray]

R. David Murray  added the comment:

Alexander, I agree with Velko in that it isn't obvious to me how the addition 
of localtime would answer the desire expressed in this issue.  It addresses 
Antoine's complaint about aware datetimes, but I don't see that it does 
anything for the "conversion to epoch based timestamp" issue.  That is at the 
very least a documentation issue, since IMO we should be providing our users 
with the tools they need to interoperate with the systems they need to 
interoperate with.

Velko: on the other hand, given Victor's research, I don't see float seconds 
since an epoch appearing anywhere as a standard.  Where do you see this being 
used as a standard?  I also don't understand your complaint about the fact that 
the one-liner creates a timetuple.  datetime stores the date and time 
information as discrete fields, so generating a timetuple is a natural 
conversion path.  

Obviously one could avoid the creation of a Python tuple by calling the C 
mktime directly in the C code, as has been proposed.  I don't see, myself, what 
would be so bad about providing a 'to_crt_timestamp' method that would, in 
essence, be the kind of light wrapper around the system API that we provide in 
so many other places in Python.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Velko Ivanov]

Velko Ivanov  added the comment:

I'm very disappointed by the outcome of this discussion.

You are committing the biggest sin of modern times - instead of promoting the 
obtaining and usage of knowledge to solve things, you place restrictions to 
force the dumbheads into not doing mistakes. The big problem with that is that 
you can never foresee all usecases and all possible mistakes, thus you will 
always be sorrily bitten by dumbheads. What does that make of you?

Let me present you a situation - I have a system that passes data via JSON, 
datetime object is not JSON serializable. For few other reasons, like the epoch 
and float secs since epoch being defacto standard, and the fact that I 
absolutely make sure at-the-source that my timestamps are UTC and lack zone 
awareness, and the fact that I'm not going to display those, but only use them 
for comparison, and that I'm not going to do historical things and calculations 
and I don't actually need nanosecond precision, just a tenth of the second, and 
I'm fine with always using the '<' and '>', not the '==', and the fact that 90% 
of the cases when use datetimes I have exactly the same requirements and it has 
always been working fine for me - I choose the lightweight float representation 
at the one side of the system.
In the SQL DB I use tz unaware timestamps, not floats and my DB access layer 
returns datetime objects and I prefer them at this end of the system. So I only 
need to serialize the datetime object. Well, as a matter of fact I have a JSON 
object serialization already in place for some of my objects, but I do not need 
that for tz unaware datetimes.
So I look for a method that makes a float from a datetime, which I'm used to in 
PHP, Java, .NET, C, SQL and you name it. And I'm 2 hours into reading about 
time, datetime and calendar modules and I still haven't even invented the 
obscure time.mktime(dt.timetuple())+dt.microseconds*1e-6 . And to even think 
that this creates a timetuple internally ? I hate it instantly and I dismiss 
the possibility that the API could be so wrong and I keep searching -> on the 
internets -> which brings me here where all my illusions are finally buried 
into the dust.

2 Hours for something, that only needs a few warning lines in the docs?
Ok, the ultimately right thing is to actually serialize the datetime object and 
rework my other end of the system to use dt instead of float .. maybe .. but 
not now - now I'm only testing an idea for something completely different and I 
only need faithful and dutiful Python to give me a float from datetime so I can 
check something.
I love Python for being simple, logical and consistent and for giving me the 
tools and not telling me what to do with them. 
Not today ... Today Python goes - 'Here is your hammer, but you can not use it 
to hit straight down. If you hit straight down, and you are using a forge, and 
you miss your object and hit the forge instead, the hammer could ricochet and 
hit you back on the forehead, so you can't use it that way. As a matter of 
fact, there is a gyroscopic sensor embedded in the handle of the hammer and if 
you try to hit with an angle that is close to 90 degrees, it will detach the 
head of the hammer from the handle and prevent you from eventually endangering 
yourself' and I'm like 'WTF??! I'm nailing a nail into a wooden plank!'

Now I'm going to use the obscure one liner and hate it, because it is simply 
wrong and only someone that doesn't care of implementation detail might think 
it equal to a proper solution.
The worst thing is, that I learned today, that if I ever need operations with 
tz aware dates and time intervals in Python, I should actually send an SQL 
query for that, because my DB has a clean, simple and COMPLETE date/time API 
that works seamlessly. Yours is a jungle and I see you being asked to include a 
ready made patch to output a float from a dt, to which you respond by adding a 
locatime() method 2 years later. 
You seriously think, that #9527 solves this? I don't even see a connection.

With #9527 in the python library I would be exactly what I am now - overly 
frustrated and with the exactly same amount of time completely lost into 
studying a bunch of tools only to realize that I should avoid using them at all 
costs.

I'm sorry if I offend somebody by posting this emotional message, I just try to 
give you another point of view - don't put restrictions and hide the reasoning. 
Instead, support the thing that is widespread and advise that in certain 
conditions there are better things to do. And if it doesn't work for some edge 
cases, or even for half the cases - place a well elaborated warning. Then if 
programmers still make the mistake - well, let them learn by it. 'Cause that's 
the way people learn .. they make mistakes. By preventing them from making the 
mistake, you actually rob them of learning.

--
nosy: +vivanov

___
Python tracker 

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
superseder:  -> Add aware local time support to datetime module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Sebastian Rittau]

Changes by Sebastian Rittau :


--
nosy:  -srittau

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, May 21, 2010 at 12:20 PM, Antoine Pitrou  wrote:
..
> Well, for example, the datetime module encourages you to use "aware" datetime 
> objects (rather than so-called "naive" objects),
> but there isn't a single facility to do so. You must reinvent a whole 
> timezone class from scratch.

This is partially addressed by issue 5094, "datetime lacks concrete
tzinfo impl. for UTC".  A more ambitious project would be to add pytz
to stdlib.  I believe I've seen this idea discussed and rejected, but
I am not able to find a link to an appropriate thread now.  A half-way
project would be to add LocalTimezone given as an example in
http://docs.python.org/dev/py3k/library/datetime.html in addition for
UTC timezone.   Any takers?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> The advantage of an obscure one-liner is
> that it is obvious what it does, particularly for someone with a
> C/UNIX background.

Well, I would argue that the C/Unix legacy in terms of dates and times isn't an 
example to follow. Python does not force you to use strcat() to concatenate 
strings, either ;)

But besides, the issue is more how people are supposed to invent that 
one-liner, let alone remember it easily. Perhaps adding it in the documentation 
would be a good middle ground, if you think it shouldn't be added to the stdlib.

> Do you have other examples of this sort?

Well, for example, the datetime module encourages you to use "aware" datetime 
objects (rather than so-called "naive" objects), but there isn't a single 
facility to do so. You must reinvent a whole timezone class from scratch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, May 21, 2010 at 11:20 AM, Alexander Belopolsky
 wrote:
..
> I believe it should be something like this:
>
> from claendar import timegm

s/claendar/calendar/, of course.

--
nosy: +Alexander.Belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
assignee:  -> belopolsky
nosy:  -Alexander.Belopolsky
stage:  -> unit test needed
versions: +Python 3.2 -Python 2.6, Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, May 21, 2010 at 7:37 AM, Antoine Pitrou  wrote:
..
> I agree with Victor that the APIs need improving, even if it involves 
> providing obvious replacements of obscure one-liners.

While I agree that the datetime API can be improved, I don't think
Victor's proposal does that.  The advantage of an obscure one-liner is
that it is obvious what it does, particularly for someone with a
C/UNIX background.  dt.totimestamp() may be easier to write, but it is
entirely non-obvious what it will return.  One would expect that
dt.totimestamp() is the inverse of datetime.fromtimestamp(timestamp),
but in timezones with daylight savings adjustments, but such inverse
may not always exist.  (01:59AM may be followed by 02:00 AM or by
01:00 AM. so on changeover days datetime(y, m, d, 1, 30).totimestamp()
is either ambiguous or undefined.)   As I suggested in my previous
comment, this problem can be resolved, but we are not there yet.

> As an occasional user of datetime and time modules, I have too often wanted 
> to curse those limited, awkwardly inconsistent APIs.

Yes, it would be ideal if a user of datetime module would not need to
reach to other modules for date/time calculations.  See also
.   Do you have other examples of
this sort?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Fri, May 21, 2010 at 7:26 AM, STINNER Victor  wrote:
..
>>  ... If the tzinfo of the datetime object does not match the
>> system TZ used by mktime, the result will be quite misleading.
>
> Can you suggest a possible fix to take care of the timezone information? I 
> don't know how to use that.

I believe it should be something like this:

from claendar import timegm
def datetime_totimestamp(dt):
return timegm(dt.utctimetuple()), dt.microsecond)

Note the following comment in the documentation for tzinfo.fromutc():
"An example of a time zone the default fromutc() implementation may
not handle correctly in all cases is one where the standard offset
(from UTC) depends on the specific date and time passed, which can
happen for political reasons.  The default implementations of
astimezone() and fromutc() may not produce the result you want if the
result is one of the hours straddling the moment the standard offset
changes."  I have not tested the above code and it may not work for
non-trivial time-zones.

Still a few questions remain:

1. Should absence of tzinfo imply local timezone or UTC?
2. Given that datetime.fromtimestamp() takes an optional tz argument,
should totimestamp() do the same and use given tz for naive datetime
objects?
3. Should there be a toutctimestamp()?

I believe at this stage we need a python implementation of a prototype
answering these questions and a unit test that would demonstrate how
the prototype would work with nontrivial timezones.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I agree with Victor that the APIs need improving, even if it involves providing 
obvious replacements of obscure one-liners. As an occasional user of datetime 
and time modules, I have too often wanted to curse those limited, awkwardly 
inconsistent APIs.

Just my 2 seconds of course :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[STINNER Victor]

STINNER Victor  added the comment:

> As you explain in your own documentation, the proposed method 
> is equivalent to ``(time.mktime(self.timetuple()), self.microsecond)``,
> so all it does is replacing a less than a one-liner.

a one-liner, but an horrible one liner :-) I don't like mixing datetime and 
time modules. I prefer to use only datetime, I prefer its API.

>  ... If the tzinfo of the datetime object does not match the 
> system TZ used by mktime, the result will be quite misleading.

Can you suggest a possible fix to take care of the timezone information? I 
don't know how to use that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Skip Montanaro]

Changes by Skip Montanaro :


--
nosy:  -skip.montanaro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Mark Dickinson]

Changes by Mark Dickinson :


--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Mark Dickinson]

Mark Dickinson  added the comment:

Close issue 1673409 as a duplicate of this one;  combining nosy lists.

--
nosy: +catlee, erik.stephens, guettli, jribbens, mark.dickinson, pitrou, 
skip.montanaro, srittau, steve.roberts, tim_one, tomster

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2736] datetime needs an "epoch" method

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Victor,

As you explain in your own documentation, the proposed method is equivalent to 
``(time.mktime(self.timetuple()), self.microsecond)``, so all it does is 
replacing a less than a one-liner.  Moreover, I am not sure 
time.mktime(self.timetuple()) is something that people would want to do with a 
TZ-aware datetime.  If the tzinfo of the datetime object does not match the 
system TZ used by mktime, the result will be quite misleading.

On the patch itself:

1. See my comment at Issue1726687 about the tm_wday == 1 typo.

2. I don't think time_t to long cast is safe on all platforms.

--
nosy: +Alexander.Belopolsky
title: datetime needs and "epoch" method -> datetime needs an "epoch" method
type: behavior -> feature request

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com