kenny75557 opened a new pull request, #59359:
URL: https://github.com/apache/spark/pull/59359
### What changes were proposed in this pull request?
This PR makes three tests in `pyspark.sql.tests.test_functions` independent
of the local timezone:
- `test_try_make_timestamp_ltz`: use the zone ID `"UTC"` instead of
`datetime.now().astimezone().tzinfo.__str__()`, which returns
an abbreviation such as `CST` or `PDT`.
- `test_make_timestamp` and `test_try_make_timestamp`: compute the
expected values for the `CET` cases from the UTC instant (08:30
UTC) instead of hardcoding `08:30`.
A small helper `_utc_to_local()` converts a UTC wall-clock time to a naive
local datetime, which is what `collect()` returns.
### Why are the changes needed?
These tests fail on any machine whose local timezone is not UTC. CI does
not catch this because the PySpark jobs run in containers that default to
**UTC**.
- Timezone abbreviations are ambiguous for Spark. With `TZ=Asia/Taipei`,
Python returns `CST`, which Java reads as US Central
Time, so 10:30 becomes 23:30. With `TZ=America/Los_Angeles` or
`America/New_York`, `PDT`/`EDT` are rejected with
`INVALID_TIMEZONE`.
- `collect()` returns timestamps in the local timezone, so the hardcoded
`08:30` (10:30 CET in UTC) only matches when the local timezone is UTC.
### Does this PR introduce _any_ user-facing change?
No, test-only change.
### How was this patch tested?
Ran the affected tests under different local timezones, in both classic
and Spark Connect mode:
```bash
for tz in UTC Asia/Taipei America/New_York America/Los_Angeles; do
TZ=$tz python/run-tests --testnames "pyspark.sql.tests.test_functions
FunctionsTests.test_make_timestamp
FunctionsTests.test_try_make_timestamp
FunctionsTests.test_try_make_timestamp_ltz"
done
TZ=Asia/Taipei python/run-tests --testnames
"pyspark.sql.tests.connect.test_parity_functions"
```
Before this change, all non-UTC runs fail; after it, all pass.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
Claude Code was used to investigate the root cause and draft the
description. I applied the code changes myself, then reviewed and verified them
locally under several timezones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
