morningman opened a new issue, #67370: URL: https://github.com/apache/doris/issues/67370
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Apache Doris 4.1.3-rc02, commit `31263df4dc1d4d3a27517d264802cd4d6b92c874` Client: Python + ADBC Flight SQL driver (`adbc_driver_flightsql`), FE `arrow_flight_sql_port` = 41070. The MySQL/JDBC protocol is used as the control path for comparison. ### What's Wrong? When reading a mapped Iceberg `TIMESTAMPTZ` column with the session `time_zone` set to `UTC`, Doris Flight SQL returns the Arrow type `timestamp[s, tz=Z]`. PyArrow treats the timezone string as a `zoneinfo` key, so `to_pylist()` raises `ZoneInfoNotFoundError` because `Z` is not a valid IANA time zone name. The underlying epoch values themselves are correct: `[1735704000, null, 939528000, null]`. Rewriting only the schema metadata from `Z` to `UTC` leaves the epochs unchanged and makes the conversion succeed. ### What You Expected? UTC timezone metadata should use a name that PyArrow / `zoneinfo` can resolve (e.g. `UTC`, or a fixed `+00:00` offset), so that `to_pylist()` returns UTC datetimes directly. ### How to Reproduce? 1. Bring up the Iceberg regression fixture and create a catalog with `"enable.mapping.timestamp_tz" = "true"`. 2. `SWITCH` to that catalog, `USE test_timestamp_tz`, and set the session `time_zone` to `UTC`. 3. Read `test_ice_timestamp_tz_orc` over Python ADBC Flight SQL. 4. Inspect the Arrow schema and call `to_pylist()`. ```sql SWITCH test_iceberg_timestamp_tz_with_mapping; USE test_timestamp_tz; SET time_zone = 'UTC'; SELECT * FROM test_ice_timestamp_tz_orc ORDER BY id; ``` Client side: ```python import adbc_driver_flightsql.dbapi as flight_sql conn = flight_sql.connect(uri="grpc://127.0.0.1:41070", db_kwargs={"username": "root", "password": ""}) cur = conn.cursor() cur.execute("SET time_zone = 'UTC'") cur.execute("SELECT * FROM test_iceberg_timestamp_tz_with_mapping." "test_timestamp_tz.test_ice_timestamp_tz_orc ORDER BY id") table = cur.fetch_arrow_table() print(table.schema) # timestamp[s, tz=Z] print(table.to_pylist()) # ZoneInfoNotFoundError ``` ### Anything Else? `Z` is valid ISO-8601 offset syntax but is not a portable Arrow timezone string; the Arrow spec expects either an IANA name or a fixed `+HH:MM` offset. The Doris session `time_zone` value is apparently passed through to the Arrow field metadata verbatim. **Workaround:** on the client, cast only the Arrow timezone metadata from `Z` to `UTC` before converting to Python; this does not change the epoch values. Casting to `STRING` in SQL also works, but loses the native `TIMESTAMPTZ` Arrow type. Found with the `external_table_p0/iceberg/test_iceberg_timestamp_tz` fixture and confirmed with a raw Arrow probe plus a metadata-only cast. Tracking issue: #65615 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
