This is an automated email from the ASF dual-hosted git repository.
Jefffrey pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new 2166741578 Resolve DST-ambiguous and nonexistent local times when
casting timestamps to a named timezone (#11038)
2166741578 is described below
commit 2166741578fed12de6133cb29f0223456fd32123
Author: Adrian Garcia Badaracco <[email protected]>
AuthorDate: Tue Sep 29 04:17:18 2026 -0500
Resolve DST-ambiguous and nonexistent local times when casting timestamps
to a named timezone (#11038)
# Which issue does this PR close?
- Closes #11037.
# Rationale for this change
## What a user hits today
Cast a naive timestamp to a named timezone. Pick a wall clock reading
that the zone skips or repeats. The cast fails.
```rust
use arrow_array::TimestampSecondArray;
use arrow_cast::{cast_with_options, CastOptions};
use arrow_schema::{DataType, TimeUnit};
// 1_710_037_800 is 2024-03-10T02:30:00, held as a naive wall clock reading.
// America/New_York skips that hour: the clocks go forward at 02:00.
let array = TimestampSecondArray::from(vec![1_710_037_800]);
let to = DataType::Timestamp(TimeUnit::Second,
Some("America/New_York".into()));
let strict = CastOptions { safe: false, ..Default::default() };
let lenient = CastOptions { safe: true, ..Default::default() };
cast_with_options(&array, &to, &strict);
cast_with_options(&array, &to, &lenient);
```
Today:
```text
strict -> Err("Cast error: Cannot cast timezone to different timezone")
lenient -> Ok, null_count = 1
```
After this PR:
```text
strict -> Ok, value = 1710055800 (2024-03-10T07:30:00Z, i.e. 03:30 EDT)
lenient -> Ok, null_count = 0
```
The `safe: true` result is the dangerous one. A whole column of
DST-boundary readings turns to NULL, and nothing reports it.
## How this reaches a SQL user
A DataFusion user reaches the same kernel through ordinary SQL. This is
how the bug was found:
```sql
SELECT '2024-03-10 02:30:00'::timestamp AT TIME ZONE 'America/New_York';
-- Cast error: Cannot cast timezone to different timezone
```
The engine plans that as `Timestamp(_, None)` to `Timestamp(_,
Some("America/New_York"))`, so the error text above comes straight from
this kernel. See https://github.com/apache/datafusion/issues/25084. An
upcoming type-coercion change inserts the same cast automatically for
`timestamptz - timestamp`, so the failure gets more common, not less.
## Why the kernel fails
`adjust_timestamp_to_timezone` reads each value as a wall clock time in
the target zone. In an IANA timezone that reading is not always one
instant:
* A "fall back" transition repeats an hour, so the reading is
**ambiguous**.
* A "spring forward" transition skips an hour, so the reading is
**nonexistent**.
The kernel resolved the offset with
`offset_from_local_datetime(..).single()`. That is `None` for both
cases, so both cases fail.
# What changes are included in this PR?
A new private helper, `resolve_local_offset`, decides the two cases
instead of rejecting them. It follows PostgreSQL and DuckDB:
* An **ambiguous** reading takes the **later** instant. This is the
offset in effect after the transition. `2024-11-03T01:30:00` in
`America/New_York` becomes `01:30-05:00` (EST).
* A **nonexistent** reading moves **forward** by the length of the gap.
`2024-03-10T02:30:00` in `America/New_York` becomes `03:30-04:00` (EDT).
`adjust_timestamp_to_timezone` calls the helper in place of `.single()`.
That is the whole behavioural change.
The forward shift needs the offset in effect *before* the transition.
The code recovers it with one probe: it asks for the offset 24 hours
earlier and takes the earliest result. `offset_from_utc_datetime` is not
a valid shortcut, because it returns the post-transition offset. That is
wrong for a southern hemisphere gap such as `Australia/Sydney`
`2024-10-06T02:30:00`.
Two properties of the timezone database make the probe safe:
* No two transitions in any of the 597 zones are closer than **167
hours**. So the probe lands on the correct side of the transition. The
smallest interval is `America/Boa_Vista` in 2000.
* No local gap is longer than **24 hours**. So the probe lands outside
the gap at all. Seven zones sit exactly at 24 hours — the dateline
changes, such as `Pacific/Apia` in 2011. The next largest gap is 10
hours.
If the probe still fails, the helper returns `None` and the caller keeps
the old error or NULL. With current tzdata that branch is unreachable.
Nothing else changes. Unit conversion, `safe` handling and the error
message stay as they were. A fixed-offset timezone has no transitions,
so it is unaffected.
# Are these changes tested?
Yes. Six new tests sit next to the existing
`test_cast_timestamp_with_timezone_*` tests in
`arrow-cast/src/cast/mod.rs`:
* `test_cast_timestamp_to_named_timezone_dst` — `America/New_York`, with
an unambiguous, an ambiguous and a nonexistent reading plus a null,
under `safe: false`.
* `test_cast_timestamp_to_named_timezone_dst_safe` — the same input
under `safe: true`, which no longer produces nulls.
* `test_cast_timestamp_to_named_timezone_dst_southern_hemisphere` —
`Australia/Sydney`, where the transitions run the other way round.
* `test_cast_timestamp_to_named_timezone_dst_nanosecond` and
`..._dst_changing_unit` — the resolution composes with the unit paths.
* `test_cast_timestamp_to_fixed_offset_timezone_unaffected` — `+08:00`
on the same readings is unchanged.
Expected values come from `chrono`, built from the wall clock reading
and the expected offset. They are not hardcoded. Revert the one-line
kernel change and the five DST tests fail, while the fixed-offset test
still passes.
## Validation
The change was checked against 97,162 cases. These cover every
transition of every IANA zone in 1890–1995 and 2023–2025: 15,871 in a
gap and 14,370 ambiguous. There were zero disagreements with an
independent implementation on the same tzdata.
Offsets resolve **per row**. One array that straddles both 2024
`America/New_York` transitions yields three distinct offsets, and every
element matches PostgreSQL 17.11.
One unrelated discrepancy came up. chrono-tz 0.10.4 bundles tzdata
**2025b**, which puts the `Europe/Chisinau` and `Europe/Tiraspol`
transitions an hour away from PostgreSQL, ICU and CPython. That
reproduces on `main` without this PR. It needs a chrono-tz bump, not a
change here.
## A note on the test manifest
arrow-cast had no way to name an IANA zone in its own tests. `Tz` parses
IANA names only when `arrow-array` is built with `chrono-tz`, and
arrow-cast never enabled it.
This PR enables it on the **dev-dependency only**: `arrow-array = {
workspace = true, features = ["chrono-tz"] }` under
`[dev-dependencies]`. The new tests then run in every arrow-cast CI job.
No public feature appears, and nothing changes for downstream crates.
One test needed an update. `test_cast_string_to_timestamp_invalid_tz`
asserted the exact error tail `only offset based timezones supported
without chrono-tz feature`, which is now `failed to parse timezone`. It
asserts the stable prefix `Parser error: Invalid timezone \"ZZTOP\":`
instead.
Local runs that pass: `cargo test -p arrow-cast` (default and
`--all-features`, debug and release), `cargo test -p arrow --features
chrono-tz,prettyprint --test array_cast --test timezone`, clippy with
`-D warnings` on the CI feature combinations, and `cargo doc
--all-features` with `-D warnings`.
# Field research: PostgreSQL 17 and DuckDB
The resolution policy is the one arbitrary decision in this change, so
it needs more than one reference. Both engines were measured directly:
PostgreSQL 17.11 in `postgres:17`, and the DuckDB 1.5.2 CLI with the ICU
extension. All times below are UTC.
| Case | PostgreSQL 17.11 | DuckDB 1.5.2 | This PR |
| --- | --- | --- | --- |
| `America/New_York` `2024-11-03 01:30` — **ambiguous** | `2024-11-03
06:30:00` | `2024-11-03 06:30:00` | `2024-11-03 06:30:00` |
| `America/Havana` `2024-11-03 00:00` — **ambiguous** local midnight |
`2024-11-03 05:00:00` | `2024-11-03 05:00:00` | `2024-11-03 05:00:00` |
| `America/New_York` `2024-03-10 02:30` — gap | `2024-03-10 07:30:00` |
`2024-03-10 07:30:00` | `2024-03-10 07:30:00` |
| `America/Sao_Paulo` `2018-11-04 00:00` — local midnight does not exist
| `2018-11-04 03:00:00` | `2018-11-04 03:00:00` | `2018-11-04 03:00:00`
|
| `Australia/Sydney` `2024-10-06 02:30` — southern hemisphere gap |
`2024-10-05 16:30:00` | `2024-10-05 16:30:00` | `2024-10-05 16:30:00` |
| `Australia/Lord_Howe` `2024-10-06 02:15` — 30 minute DST step |
`2024-10-05 15:45:00` | `2024-10-05 15:45:00` | `2024-10-05 15:45:00` |
| `Pacific/Chatham` `2024-09-29 03:00` — +12:45 / +13:45 | `2024-09-28
14:15:00` | `2024-09-28 14:15:00` | `2024-09-28 14:15:00` |
**The two ambiguous rows carry the argument.** A gap has only one
sensible answer, so it discriminates nothing. An ambiguous reading has
two real answers, and the choice between them is the policy.
`2024-11-03 01:30` in New York is either `05:30Z` (EDT) or `06:30Z`
(EST). Both engines give `06:30Z`. Havana's ambiguous midnight is either
`04:00Z` (CDT) or `05:00Z` (CST). Both engines give `05:00Z`. In both
cases the answer is **the later instant**, which is what this PR does.
## Spark takes the earlier instant
Spark 4.2.0 was measured on the same cases. `CAST(TIMESTAMP_NTZ ... AS
TIMESTAMP)`, `to_utc_timestamp`, `convert_timezone` and a string cast
all agree, with ANSI mode on and off. Spark matches every gap row above.
It takes the **earlier** instant for every ambiguous reading: New York
`01:30` gives `05:30Z`, Havana midnight gives `04:00Z`, and
`Australia/Sydney` `2024-04-07 02:30` gives `2024-04-06 15:30Z`. Spark
follows Java here: the cast calls
[`convertTz`](https://github.com/apache/spark/blob/b1b2685058d6359e00842cbd4c723f4ce3810709/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L901),
which uses `LocalDateTime.atZone` (`ZonedDateTime.of`). This PR keeps
the PostgreSQL and DuckDB rule.
## The case neither engine can arbitrate
A fixed-offset string such as `'+05:30'` has no agreed reading, so it is
recorded here rather than hidden. Measured:
| Spelling | PostgreSQL 17.11 | Convention |
| --- | --- | --- |
| `SET TimeZone='+05:30'` then `'2024-01-01 12:00:00'::timestamptz` |
`17:30Z` | POSIX, west-positive |
| `'2024-01-01 12:00:00'::timestamp AT TIME ZONE '+05:30'` | `17:30Z` |
POSIX, west-positive |
| `timestamptz '2024-01-01 12:00:00 +05:30'` | `06:30Z` | ISO,
east-positive |
PostgreSQL agrees with arrow-rs when the offset sits inside the literal.
It disagrees when the offset acts as a zone name. DuckDB 1.5.2 rejects
the zone-name spelling outright: `Not implemented Error: Unknown
TimeZone '+05:30'`.
So neither engine settles it. This PR does not change that behaviour. It
is tracked downstream in
https://github.com/apache/datafusion/issues/25170.
# Are there any user-facing changes?
Yes. A cast that previously raised an error, or returned NULL under
`safe: true`, now returns the instants described above. There is no API
change.
## This PR does not close DataFusion #25084 on its own
There are two independent paths to a DST-boundary failure. This PR fixes
one of them:
```sql
-- cast kernel, fixed here
SELECT '2024-03-10 02:30:00'::timestamp AT TIME ZONE 'America/New_York';
-- string parser, NOT fixed here (see #11039)
SELECT '2024-03-10 02:30:00' AT TIME ZONE 'America/New_York';
SET datafusion.execution.time_zone = 'America/New_York';
SELECT '2024-03-10 02:30:00'::timestamptz;
-- Parser error: Error parsing timestamp from '2024-03-10 02:30:00': error
computing timezone offset
```
To a user those are the same query, and the last spelling is the most
natural one. Merge this alone and an explicit `::timestamp` in the
middle of an expression makes a query start to work. That is harder to
explain than the current uniform failure. #11039 must land with this PR
or close behind it, and must use the same policy so the two paths cannot
drift.
This PR **is** sufficient for
https://github.com/apache/datafusion/issues/10308, which is entirely on
the cast path.
## Follow-up: `arrow-array` resolves ambiguity the opposite way
After this PR, arrow resolves an ambiguous local time two different ways
in two crates:
| | Ambiguous | In a gap |
| --- | --- | --- |
| `arrow-cast::resolve_local_offset` (this PR) | the **later** instant |
offset probed 24 hours earlier |
| `arrow-array::types::from_naive_datetime`
([`types.rs:347`](https://github.com/apache/arrow-rs/blob/main/arrow-array/src/types.rs#L347))
| the **earlier** instant (`Ambiguous(dt1, _)`) | `None` |
That function is deliberately untouched here.
This matters downstream. DataFusion calls the `arrow-array` form
directly with a `Some(tz)` in
`datafusion/functions/src/datetime/date_part.rs` (`date_to_scalar`, four
call sites). It builds local-midnight bounds for the `date_part(YEAR,
col) = <year>` rewrite. Where that local midnight is ambiguous, the
earlier instant becomes an **upper** bound, so valid rows can drop out.
The behaviour predates this PR, but this PR widens the gap between the
two policies.
The two want reconciling, or the difference wants documenting on both
functions. Changing `from_naive_datetime` affects every caller of
`arrow-array`, so it deserves its own review rather than a ride along
with a cast fix.
## Follow-up: a loud error becomes a quiet non-identity
`to_local_time(ts) AT TIME ZONE tz` is a documented DataFusion idiom.
Across an ambiguous hour it is no longer the identity, and it no longer
says so. Before this PR the second leg raised `Cannot cast timezone to
different timezone`:
```text
orig local roundtrip
identity
2021-10-31T02:00:00+02:00 2021-10-31T02:00:00 2021-10-31T02:00:00+01:00
false
2021-10-31T02:30:00+02:00 2021-10-31T02:30:00 2021-10-31T02:30:00+01:00
false
2021-10-31T02:00:00+01:00 2021-10-31T02:00:00 2021-10-31T02:00:00+01:00
true
```
PostgreSQL behaves the same way. `t = ((t AT TIME ZONE
'Europe/Brussels') AT TIME ZONE 'Europe/Brussels')` is `f` for those
same two instants. So this is the intended trade, not an argument
against the change. But it is a real behaviour change, and downstream
docs want to say so.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.1 <[email protected]>
Co-authored-by: Jeffrey Vo <[email protected]>
---
arrow-cast/Cargo.toml | 3 +
arrow-cast/src/cast/mod.rs | 270 +++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 265 insertions(+), 8 deletions(-)
diff --git a/arrow-cast/Cargo.toml b/arrow-cast/Cargo.toml
index 4880958195..eab99b59ad 100644
--- a/arrow-cast/Cargo.toml
+++ b/arrow-cast/Cargo.toml
@@ -55,6 +55,9 @@ base64 = "0.23"
ryu = "1.0.16"
[dev-dependencies]
+# Enable `chrono-tz` so that tests can exercise IANA (named) timezones such as
+# `America/New_York`, which is where DST transitions can be observed.
+arrow-array = { workspace = true, features = ["chrono-tz"] }
criterion = { workspace = true, default-features = false }
half = { version = "2.1", default-features = false }
insta = { workspace = true }
diff --git a/arrow-cast/src/cast/mod.rs b/arrow-cast/src/cast/mod.rs
index 96b9e4764c..1367338405 100644
--- a/arrow-cast/src/cast/mod.rs
+++ b/arrow-cast/src/cast/mod.rs
@@ -57,7 +57,9 @@ pub use crate::cast::union::*;
use arrow_buffer::IntervalMonthDayNano;
use arrow_data::ByteView;
-use chrono::{NaiveTime, Offset, TimeZone, Utc};
+use chrono::{
+ FixedOffset, LocalResult, NaiveDateTime, NaiveTime, Offset, TimeDelta,
TimeZone, Utc,
+};
use std::cmp::Ordering;
use std::sync::Arc;
@@ -1921,6 +1923,11 @@ pub fn cast_with_options(
// unchanged.
//
// i.e. Timestamp('2001-01-01T00:00', None) ->
Timestamp('2001-01-01T00:00', '+0700')
+ //
+ // For IANA timezones a wall clock reading is not always a
unique
+ // instant: a "fall back" DST transition makes an hour occur
twice
+ // and a "spring forward" transition skips an hour entirely.
See
+ // `resolve_local_offset` for how those readings are resolved.
(None, Some(to_tz)) => {
let to_tz: Tz = to_tz.parse()?;
match to_unit {
@@ -2600,6 +2607,65 @@ fn cast_numeric_to_binary<FROM: ArrowPrimitiveType, O:
OffsetSizeTrait>(
)?))
}
+/// Returns the offset to use when interpreting `local` as a wall clock reading
+/// in `tz`, or `None` if it cannot be resolved.
+///
+/// `None` is not expected in practice. With the current timezone database no
+/// reading reaches it, because every ambiguous or nonexistent reading resolves
+/// as described below. The `None` path is a safeguard against a future
+/// timezone database that breaks the assumptions of the gap handling. Callers
+/// then apply their usual error or null handling.
+///
+/// In an IANA timezone a wall clock reading does not always identify a unique
+/// instant, and this function picks one following the same rules as PostgreSQL
+/// and DuckDB:
+///
+/// * **Ambiguous** -- when the clocks go back ("fall back") the same reading
+/// occurs twice. The *later* instant is chosen, i.e. the offset in effect
+/// after the transition. For example `2024-11-03T01:30:00` in
+/// `America/New_York` is read as `-05:00` (EST), not `-04:00` (EDT).
+/// * **Nonexistent** -- when the clocks go forward ("spring forward") the
+/// reading never occurs. It is shifted forward by the length of the gap,
+/// which is the same as reading it with the offset in effect *before* the
+/// transition. For example `2024-03-10T02:30:00` in `America/New_York` is
+/// read as `-05:00` (EST) and therefore denotes `2024-03-10T03:30:00-04:00`.
+///
+/// Timezones with a fixed offset are never ambiguous and have no gaps.
+///
+/// See <https://github.com/apache/arrow-rs/issues/11037> for the PostgreSQL
and
+/// ICU (DuckDB) sources these rules are taken from.
+fn resolve_local_offset(tz: &Tz, local: &NaiveDateTime) -> Option<FixedOffset>
{
+ match tz.offset_from_local_datetime(local) {
+ LocalResult::Single(offset) => Some(offset.fix()),
+ LocalResult::Ambiguous(_earlier, later) => Some(later.fix()),
+ LocalResult::None => {
+ // The reading falls in a gap. Recover the offset in effect before
+ // the transition by probing 24 hours earlier.
+ //
+ // Two separate properties of the timezone database make this
sound:
+ //
+ // 1. No local gap is longer than 24 hours, so the probe lands
+ // outside this gap and is itself resolvable. Seven zones have a
+ // gap of exactly 24 hours -- the dateline changes, such as
+ // `Pacific/Apia` in 2011 and `Pacific/Kiritimati` in 1994. At
the
+ // last second of one of those the probe lands one second before
+ // the gap starts, so the true margin here is one second, not a
+ // comfortable one.
+ // 2. No two transitions are closer together than 24 hours, so the
+ // offset the probe finds is the one in effect immediately
before
+ // this transition, and not some older offset. The smallest
+ // observed interval is 167 hours (`America/Boa_Vista`, 2000).
+ //
+ // Property 1 is what makes the probe resolvable; property 2 is
what
+ // makes the answer correct. If the probe is still unresolvable,
give
+ // up and let the caller apply the usual error / null handling.
+ tz.offset_from_local_datetime(&(*local - TimeDelta::hours(24)))
+ .earliest()
+ .map(|offset| offset.fix())
+ }
+ }
+}
+
fn adjust_timestamp_to_timezone<T: ArrowTimestampType>(
array: PrimitiveArray<Int64Type>,
to_tz: &Tz,
@@ -2607,8 +2673,8 @@ fn adjust_timestamp_to_timezone<T: ArrowTimestampType>(
) -> Result<PrimitiveArray<Int64Type>, ArrowError> {
let adjust = |o| {
let local = as_datetime::<T>(o)?;
- let offset = to_tz.offset_from_local_datetime(&local).single()?;
- T::from_naive_datetime(local - offset.fix(), None)
+ let offset = resolve_local_offset(to_tz, &local)?;
+ T::from_naive_datetime(local - offset, None)
};
let adjusted = if cast_options.safe {
array.unary_opt::<_, Int64Type>(adjust)
@@ -6956,6 +7022,191 @@ mod tests {
assert!(result.is_null(2));
}
+ /// The i64 value of a `Timestamp(Second, None)` holding the given wall
clock
+ /// reading (a naive timestamp is stored as if it were UTC).
+ fn naive_seconds(y: i32, m: u32, d: u32, h: u32, min: u32) -> i64 {
+ NaiveDate::from_ymd_opt(y, m, d)
+ .unwrap()
+ .and_hms_opt(h, min, 0)
+ .unwrap()
+ .and_utc()
+ .timestamp()
+ }
+
+ /// The instant, in seconds since the epoch, denoted by the given wall
clock
+ /// reading at a fixed offset of `offset_hours`.
+ fn instant_seconds(y: i32, m: u32, d: u32, h: u32, min: u32, offset_hours:
i32) -> i64 {
+ let offset = FixedOffset::east_opt(offset_hours * 3600).unwrap();
+ NaiveDate::from_ymd_opt(y, m, d)
+ .unwrap()
+ .and_hms_opt(h, min, 0)
+ .unwrap()
+ .and_local_timezone(offset)
+ .unwrap()
+ .timestamp()
+ }
+
+ // Cast Timestamp(_, None) -> Timestamp(_, Some(IANA timezone)) across DST
+ // transitions. See `resolve_local_offset`.
+ #[test]
+ fn test_cast_timestamp_to_named_timezone_dst() {
+ // Unambiguous, EDT (-04:00) is in effect.
+ let unambiguous = naive_seconds(2024, 11, 1, 0, 0);
+ // Ambiguous: the clocks go back at 2024-11-03T02:00 EDT, so 01:30
+ // happens twice, first at -04:00 and then at -05:00.
+ let ambiguous = naive_seconds(2024, 11, 3, 1, 30);
+ // Nonexistent: the clocks go forward at 2024-03-10T02:00 EST, so 02:30
+ // never happens.
+ let nonexistent = naive_seconds(2024, 3, 10, 2, 30);
+ assert_eq!(
+ [unambiguous, ambiguous, nonexistent],
+ [1_730_419_200, 1_730_597_400, 1_710_037_800]
+ );
+
+ let array = TimestampSecondArray::from(vec![
+ Some(unambiguous),
+ Some(ambiguous),
+ Some(nonexistent),
+ None,
+ ]);
+ let to_type = DataType::Timestamp(TimeUnit::Second,
Some("America/New_York".into()));
+ let options = CastOptions {
+ safe: false,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampSecondType>();
+
+ assert_eq!(c.value(0), instant_seconds(2024, 11, 1, 0, 0, -4));
+ assert_eq!(c.value(0), 1_730_433_600);
+ // The later of the two candidates, i.e. EST rather than EDT.
+ assert_eq!(c.value(1), instant_seconds(2024, 11, 3, 1, 30, -5));
+ assert_eq!(c.value(1), 1_730_615_400);
+ // Shifted forward by the one hour gap: 02:30 EST is 03:30 EDT.
+ assert_eq!(c.value(2), instant_seconds(2024, 3, 10, 2, 30, -5));
+ assert_eq!(c.value(2), instant_seconds(2024, 3, 10, 3, 30, -4));
+ assert_eq!(c.value(2), 1_710_055_800);
+ assert!(c.is_null(3));
+ }
+
+ // The same values must not become null when `safe` casting is requested.
+ #[test]
+ fn test_cast_timestamp_to_named_timezone_dst_safe() {
+ let array = TimestampSecondArray::from(vec![
+ Some(naive_seconds(2024, 11, 1, 0, 0)),
+ Some(naive_seconds(2024, 11, 3, 1, 30)),
+ Some(naive_seconds(2024, 3, 10, 2, 30)),
+ None,
+ ]);
+ let to_type = DataType::Timestamp(TimeUnit::Second,
Some("America/New_York".into()));
+ let options = CastOptions {
+ safe: true,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampSecondType>();
+ assert_eq!(c.null_count(), 1);
+ assert_eq!(c.value(0), 1_730_433_600);
+ assert_eq!(c.value(1), 1_730_615_400);
+ assert_eq!(c.value(2), 1_710_055_800);
+ assert!(c.is_null(3));
+ }
+
+ // Southern hemisphere: the transitions run the other way around.
+ #[test]
+ fn test_cast_timestamp_to_named_timezone_dst_southern_hemisphere() {
+ // Ambiguous: clocks go back at 2024-04-07T03:00 AEDT (+11:00 ->
+10:00).
+ let ambiguous = naive_seconds(2024, 4, 7, 2, 30);
+ // Nonexistent: clocks go forward at 2024-10-06T02:00 AEST (+10:00 ->
+11:00).
+ let nonexistent = naive_seconds(2024, 10, 6, 2, 30);
+
+ let array = TimestampSecondArray::from(vec![Some(ambiguous),
Some(nonexistent)]);
+ let to_type = DataType::Timestamp(TimeUnit::Second,
Some("Australia/Sydney".into()));
+ let options = CastOptions {
+ safe: false,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampSecondType>();
+
+ // The later of the two candidates, i.e. AEST (+10:00).
+ assert_eq!(c.value(0), instant_seconds(2024, 4, 7, 2, 30, 10));
+ assert_eq!(c.value(0), 1_712_421_000);
+ // Shifted forward by the one hour gap: 02:30 AEST is 03:30 AEDT.
+ assert_eq!(c.value(1), instant_seconds(2024, 10, 6, 2, 30, 10));
+ assert_eq!(c.value(1), instant_seconds(2024, 10, 6, 3, 30, 11));
+ assert_eq!(c.value(1), 1_728_145_800);
+ }
+
+ // The resolution is independent of the time unit.
+ #[test]
+ fn test_cast_timestamp_to_named_timezone_dst_nanosecond() {
+ let ambiguous = naive_seconds(2024, 11, 3, 1, 30) * 1_000_000_000 +
123_456_789;
+ let nonexistent = naive_seconds(2024, 3, 10, 2, 30) * 1_000_000_000 +
123_456_789;
+ let array = TimestampNanosecondArray::from(vec![Some(ambiguous),
Some(nonexistent)]);
+ let to_type = DataType::Timestamp(TimeUnit::Nanosecond,
Some("America/New_York".into()));
+ let options = CastOptions {
+ safe: false,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampNanosecondType>();
+ assert_eq!(c.value(0), 1_730_615_400 * 1_000_000_000 + 123_456_789);
+ assert_eq!(c.value(1), 1_710_055_800 * 1_000_000_000 + 123_456_789);
+ }
+
+ // Unit conversion still composes with the timezone adjustment.
+ #[test]
+ fn test_cast_timestamp_to_named_timezone_dst_changing_unit() {
+ let array = TimestampSecondArray::from(vec![
+ Some(naive_seconds(2024, 11, 3, 1, 30)),
+ Some(naive_seconds(2024, 3, 10, 2, 30)),
+ ]);
+ let to_type = DataType::Timestamp(TimeUnit::Millisecond,
Some("America/New_York".into()));
+ let options = CastOptions {
+ safe: false,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampMillisecondType>();
+ assert_eq!(c.value(0), 1_730_615_400_000);
+ assert_eq!(c.value(1), 1_710_055_800_000);
+ }
+
+ // A fixed offset has no transitions, so the same readings are unaffected.
+ #[test]
+ fn test_cast_timestamp_to_fixed_offset_timezone_unaffected() {
+ let array = TimestampSecondArray::from(vec![
+ Some(naive_seconds(2024, 11, 1, 0, 0)),
+ Some(naive_seconds(2024, 11, 3, 1, 30)),
+ Some(naive_seconds(2024, 3, 10, 2, 30)),
+ None,
+ ]);
+ let to_type = DataType::Timestamp(TimeUnit::Second,
Some("+08:00".into()));
+ let options = CastOptions {
+ safe: false,
+ ..Default::default()
+ };
+
+ let b = cast_with_options(&array, &to_type, &options).unwrap();
+ assert_eq!(b.data_type(), &to_type);
+ let c = b.as_primitive::<TimestampSecondType>();
+ assert_eq!(c.value(0), instant_seconds(2024, 11, 1, 0, 0, 8));
+ assert_eq!(c.value(1), instant_seconds(2024, 11, 3, 1, 30, 8));
+ assert_eq!(c.value(2), instant_seconds(2024, 3, 10, 2, 30, 8));
+ assert!(c.is_null(3));
+ }
+
#[test]
fn test_cast_date64_to_timestamp() {
let array = Date64Array::from(vec![Some(864000000005),
Some(1545696000001), None]);
@@ -12598,11 +12849,14 @@ mod tests {
};
for dt in data_types {
- assert_eq!(
- cast_with_options(&array, &dt, &cast_options)
- .unwrap_err()
- .to_string(),
- "Parser error: Invalid timezone \"ZZTOP\": only offset based
timezones supported without chrono-tz feature"
+ // The trailing detail of the message differs depending on whether
+ // `chrono-tz` is enabled, so only the common prefix is asserted.
+ let err = cast_with_options(&array, &dt, &cast_options)
+ .unwrap_err()
+ .to_string();
+ assert!(
+ err.starts_with("Parser error: Invalid timezone \"ZZTOP\":"),
+ "{err}"
);
}
}
