This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fury.git
The following commit(s) were added to refs/heads/main by this push:
new 05bfc6e7 fix(python): TimestampSerializer fails on Windows for naive
datetimes near epoch (#2209)
05bfc6e7 is described below
commit 05bfc6e7ac7fbc3a9a364c57779c3d210750e2ae
Author: LouShaokun <[email protected]>
AuthorDate: Wed May 7 21:39:16 2025 +0800
fix(python): TimestampSerializer fails on Windows for naive datetimes near
epoch (#2209)
<!--
**Thanks for contributing to Fury.**
**If this is your first time opening a PR on fury, you can refer to
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).**
Contribution Checklist
- The **Apache Fury (incubating)** community has restrictions on the
naming of pr titles. You can also find instructions in
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).
- Fury has a strong focus on performance. If the PR you submit will have
an impact on performance, please benchmark it first and provide the
benchmark result here.
-->
## What does this PR do?
This PR resolves an `OSError: [Errno 22] Invalid argument` in
`TimestampSerializer` on Windows when serializing naive `datetime`
objects (no `tzinfo`) close to the Unix epoch. This issue stems from
`datetime.datetime.timestamp()` failing for such dates on Windows.
The fix applies a workaround for Windows when handling naive `datetime`
objects:
1. The naive `datetime` is made timezone-aware by setting its `tzinfo`
to `datetime.timezone.utc`.
2. The POSIX timestamp is then retrieved.
3. This timestamp is adjusted by the local timezone offset (considering
DST) to produce the correct microsecond timestamp for serialization.
This approach ensures correct timestamp generation on Windows for these
edge cases without affecting other platforms or timezone-aware
`datetime` objects. The fix is implemented in both `_serialization.pyx`
and `_serializer.py`.
## Related issues
- #2118
## Does this PR introduce any user-facing change?
<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
---
python/pyfury/_serialization.pyx | 19 +++++++++++++++++--
python/pyfury/_serializer.py | 15 +++++++++++++--
2 files changed, 30 insertions(+), 4 deletions(-)
diff --git a/python/pyfury/_serialization.pyx b/python/pyfury/_serialization.pyx
index 4103df15..226f4547 100644
--- a/python/pyfury/_serialization.pyx
+++ b/python/pyfury/_serialization.pyx
@@ -22,6 +22,8 @@
import datetime
import logging
import os
+import platform
+import time
import warnings
from typing import TypeVar, Union, Iterable
@@ -1189,14 +1191,27 @@ cdef class
DateSerializer(CrossLanguageCompatibleSerializer):
@cython.final
cdef class TimestampSerializer(CrossLanguageCompatibleSerializer):
+ cdef bint win_platform
+
+ def __init__(self, fury, type_: Union[type, TypeVar]):
+ super().__init__(fury, type_)
+ self.win_platform = platform.system() == "Windows"
+
+ cdef inline _get_timestamp(self, value):
+ seconds_offset = 0
+ if self.win_platform and value.tzinfo is None:
+ is_dst = time.daylight and time.localtime().tm_isdst > 0
+ seconds_offset = time.altzone if is_dst else time.timezone
+ value = value.replace(tzinfo=datetime.timezone.utc)
+ return int((value.timestamp() + seconds_offset) * 1000000)
+
cpdef inline write(self, Buffer buffer, value):
if type(value) is not datetime.datetime:
raise TypeError(
"{} should be {} instead of {}".format(value, datetime,
type(value))
)
# TimestampType represent micro seconds
- timestamp = int(value.timestamp() * 1000000)
- buffer.write_int64(timestamp)
+ buffer.write_int64(self._get_timestamp(value))
cpdef inline read(self, Buffer buffer):
ts = buffer.read_int64() / 1000000
diff --git a/python/pyfury/_serializer.py b/python/pyfury/_serializer.py
index 31ac32cd..4b64ee8f 100644
--- a/python/pyfury/_serializer.py
+++ b/python/pyfury/_serializer.py
@@ -17,6 +17,8 @@
import datetime
import logging
+import platform
+import time
from abc import ABC, abstractmethod
from typing import Dict
@@ -194,14 +196,23 @@ class DateSerializer(CrossLanguageCompatibleSerializer):
class TimestampSerializer(CrossLanguageCompatibleSerializer):
+ __win_platform = platform.system() == "Windows"
+
+ def _get_timestamp(self, value: datetime.datetime):
+ seconds_offset = 0
+ if TimestampSerializer.__win_platform and value.tzinfo is None:
+ is_dst = time.daylight and time.localtime().tm_isdst > 0
+ seconds_offset = time.altzone if is_dst else time.timezone
+ value = value.replace(tzinfo=datetime.timezone.utc)
+ return int((value.timestamp() + seconds_offset) * 1000000)
+
def write(self, buffer, value: datetime.datetime):
if not isinstance(value, datetime.datetime):
raise TypeError(
"{} should be {} instead of {}".format(value, datetime,
type(value))
)
# TimestampType represent micro seconds
- timestamp = int(value.timestamp() * 1000000)
- buffer.write_int64(timestamp)
+ buffer.write_int64(self._get_timestamp(value))
def read(self, buffer):
ts = buffer.read_int64() / 1000000
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
