kz930 opened a new issue, #9074:
URL: https://github.com/apache/texera/issues/9074
### What happened?
A timestamp before 1883 comes out of a workflow's result storage minutes
away from the value the operator produced. On a machine set to
America/Los_Angeles, `1600-06-15 08:30:00` is stored as `1600-06-15 08:37:02`.
`1677-09-21 00:00:00` becomes `00:07:02`. A timestamp from 2024 is unchanged.
`IcebergUtil` writes a TIMESTAMP field as
`ts.toInstant.atZone(ZoneId.systemDefault()).toLocalDateTime` and reads it back
with `Timestamp.valueOf(ldt)`. The two halves of the write follow different
zone rules. `java.sql.Timestamp` turns its wall clock into an instant with
`java.util.Date`'s rules, which give a date before 1900 the zone's standard
offset, -8:00. `java.time` turns that instant back into a wall clock with the
zone's full history, which before 1883 is local mean time, -7:52:58. The 7
minutes 2 seconds between them is what the stored value gains. The size of the
shift depends on the machine's zone.
Writing the field as `ts.toLocalDateTime` keeps the wall clock, since
`Timestamp.valueOf` is its inverse.
### How to reproduce?
On a machine set to America/Los_Angeles, the write step alone shows it in
`jshell`:
```java
var ts = java.sql.Timestamp.valueOf("1600-06-15 08:30:00");
ts.toInstant().atZone(java.time.ZoneId.systemDefault()).toLocalDateTime();
// 1600-06-15T08:37:02
ts.toLocalDateTime();
// 1600-06-15T08:30
```
End to end, a workflow that reads a CSV with `1600-06-15 08:30:00` in a
timestamp column and passes it through Distinct reads `1600-06-15 08:37:02.0`
back from its result storage, the storage the result panel reads.
### Version/Branch
1.4.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
3ba45ee5a
### What browsers are you seeing the problem on?
_No response_
### Relevant log output
```shell
No error is logged. The stored value is silently different.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
