Re: [I] [C++] `compute::LocalTimestamp()` Performs incorrect conversion [arrow]
gowerc commented on issue #45751:
URL: https://github.com/apache/arrow/issues/45751#issuecomment-2720501383
hmm I'm not sure to be honest. I mean on the surface it definitely appears
to be related but I'm not sure its exactly the same. In that ticket the issue
seems to be a mismatch in how arrow / python interpolate missing rules when
going into the future. Here however I can clearly see that the rules in my
local tzdata database extend up until 2499:
```
> zdump -v America/New_York
America/New_York Sun Mar 9 06:59:59 2498 UT = Sun Mar 9 01:59:59 2498 EST
isdst=0 gmtoff=-18000
America/New_York Sun Mar 9 07:00:00 2498 UT = Sun Mar 9 03:00:00 2498 EDT
isdst=1 gmtoff=-14400
America/New_York Sun Nov 2 05:59:59 2498 UT = Sun Nov 2 01:59:59 2498 EDT
isdst=1 gmtoff=-14400
America/New_York Sun Nov 2 06:00:00 2498 UT = Sun Nov 2 01:00:00 2498 EST
isdst=0 gmtoff=-18000
America/New_York Sun Mar 8 06:59:59 2499 UT = Sun Mar 8 01:59:59 2499 EST
isdst=0 gmtoff=-18000
America/New_York Sun Mar 8 07:00:00 2499 UT = Sun Mar 8 03:00:00 2499 EDT
isdst=1 gmtoff=-14400
America/New_York Sun Nov 1 05:59:59 2499 UT = Sun Nov 1 01:59:59 2499 EDT
isdst=1 gmtoff=-14400
America/New_York Sun Nov 1 06:00:00 2499 UT = Sun Nov 1 01:00:00 2499 EST
isdst=0 gmtoff=-18000
```
I also get the correct expected behaviour from both R, Python and Cpp which
as far as I can tell are all using the system tzdata source as well so they
should be consistent.
```python
import zoneinfo
import datetime
def printtime(time: int):
ny_time = datetime.datetime.fromtimestamp(time,
zoneinfo.ZoneInfo("America/New_York"))
print(f"Time: {ny_time} ({ny_time.tzname()})")
print(zoneinfo.TZPATH) # ('/usr/share/zoneinfo', '/usr/lib/zoneinfo',
'/usr/share/lib/zoneinfo', '/etc/zoneinfo')
printtime(2095940701) # Time: 2036-06-01 09:45:01-04:00 (EDT)
printtime(2127476701) # Time: 2037-06-01 09:45:01-04:00 (EDT)
printtime(2159012701) # Time: 2038-06-01 09:45:01-04:00 (EDT)
printtime(2190548701) # Time: 2039-06-01 09:45:01-04:00 (EDT)
```
```R
as.POSIXct(2095940701, tz = "America/New_York") # "2036-06-01 09:45:01
EDT"
as.POSIXct(2127476701, tz = "America/New_York") # "2037-06-01 09:45:01
EDT"
as.POSIXct(2159012701, tz = "America/New_York") # "2038-06-01 09:45:01
EDT"
as.POSIXct(2190548701, tz = "America/New_York") # "2039-06-01 09:45:01
EDT"
```
```cpp
#include
#include
#include
void printme(long long x) {
std::chrono::sys_seconds utc_time{std::chrono::seconds(x)};
std::chrono::zoned_time ny_time{"America/New_York", utc_time};
std::cout << "Local time: " << std::format("{:%F %T %Z}", ny_time) <<
'\n';
}
int main() {
std::cout << "C++ Standard Version: " << __cplusplus << std::endl; //
2020
printme(2095940701);// Local time: 2036-06-01 09:45:01 EDT
printme(2127476701);// Local time: 2037-06-01 09:45:01 EDT
printme(2159012701);// Local time: 2038-06-01 09:45:01 EDT
printme(2190548701);// Local time: 2039-06-01 09:45:01 EDT
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] [C++] `compute::LocalTimestamp()` Performs incorrect conversion [arrow]
kou commented on issue #45751: URL: https://github.com/apache/arrow/issues/45751#issuecomment-2719630226 Ah, sorry. I misunderstood this. I thought that `local_timestamp()` converts nothing but `local_timesatmp()` converts but wrong offset is used, right? Is this duplicated of https://github.com/apache/arrow/issues/36110 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [C++] `compute::LocalTimestamp()` Performs incorrect conversion [arrow]
gowerc commented on issue #45751:
URL: https://github.com/apache/arrow/issues/45751#issuecomment-2717193958
Hi @kou thank you for the reply,
> You need to convert it by yourself or we may want to add a new compute
kernel for it.
Apologies I am confused, from the documentation I thought that was the exact
purpose of the `local_timestamp()` computation ? In particular from the docs:
> local_timestamp function converts UTC-relative timestamps to local
“timezone-naive” timestamps. The timezone is taken from the timezone metadata
of the input timestamps.
At least the implication of that from the the way its written is that it is
performing the following calculation (which is what I am looking for):
$$
time_{local} = time_{utc} + offset(timezone)
$$
I should also note for ~99% of cases I've tested so far the
`local_timestamp` function appears to be working as I was hoping / expecting, I
just found this one example where it is not performing as expected.
> BTW, why do you want to get offset-ed seconds?
I am trying to write a small CLI tool that converts parquet data to XPT
format; XPT format however has no support for timezones so how to correctly
store timestamp data is dependent on the use case; some users prefer to store
the data as timezone-naive whilst others (myself included) prefer to just store
the UTC-relative timestamps. To this end I am just providing an option for the
user to choose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] [C++] `compute::LocalTimestamp()` Performs incorrect conversion [arrow]
kou commented on issue #45751: URL: https://github.com/apache/arrow/issues/45751#issuecomment-2716155773 `TimestampArray` values don't depend on timezone. `TimestampArray::type()` has timezone information instead. If you want to get offset-ed seconds by timezone. You need to convert it by yourself or we may want to add a new compute kernel for it. BTW, why do you want to get offset-ed seconds? FYI: The document of `local_timestamp()`: https://arrow.apache.org/docs/cpp/compute.html#timezone-handling -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
