OIiveirra opened a new pull request, #68161:
URL: https://github.com/apache/doris/pull/68161
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Paimon can retain historical files written with a higher timestamp precision
after the table schema evolves to a lower precision. The repair must preserve
the historical value until Doris materializes the current schema precision;
otherwise fractional seconds can be rounded, lost by the Paimon predicate, or
corrupted for timestamps before the Unix epoch.
This PR is deliberately scoped to Format V2 and JNI Paimon reads. Format V2
marks the Paimon precision narrowing as final projection work and truncates
DATETIMEV2 and TIMESTAMPTZ values without changing ordinary SQL CAST behavior.
The JNI reader widens timestamp leaves to precision 9, recursively through ROW,
ARRAY, and MAP, disables Paimon-side predicates while repair is active, and
truncates the materialized value to the Doris target precision. The fixed-width
truncation also covers leading-zero fractions and pre-epoch values.
V1 reader and V1 converter behavior is unchanged by this PR. Unrelated
Iceberg fixture and suite changes are not included.
### Release note
Paimon Format V2 and JNI scans now truncate evolved timestamp values to the
table precision.
### Check List (For Author)
- Test: targeted regression, unit, and static checks
- Personal external regression: paimon_timestamp_types, 1 suite, 0
failed, using the Paimon profile
- BE unit: ColumnTypeConverterTest, 10/10 passed
- FE unit reports: PaimonJniScannerTest, 34/34; PaimonColumnValueTest,
9/9
- Clang-format 16, build hygiene, and git diff checks passed
- Full community P0/P1, FE/BE UT, and other CI groups are delegated to
the community pipeline
- Behavior changed: Yes. Paimon Format V2 and JNI timestamp precision
evolution now truncates fractional seconds to the current table precision; V1
behavior and ordinary SQL casts are unchanged.
- Does this need documentation: No
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
