Cool, makes sense. I am def not following the lists and dev as close as I had in the past. So, good for those references.
Content with letting you run with this. It did seem there is a straightforward way to help get time mostly sorted, and gracefully evolved. So, glad you're taking steps in that direction. I'll close my PR, please continue! cheers - On Tue, Mar 17, 2026 at 5:34 AM Ahmed Abualsaud via dev <[email protected]> wrote: > Hey Austin, thanks for bringing this up! I'm +1 to this direction but > would like to provide some nuance. > > I opened a PR a few days ago to fully upgrade Date to a portable type ( > #37830 <https://github.com/apache/beam/pull/37830>), addressing the same > Iceberg issue you pointed to (#37823 > <https://github.com/apache/beam/issues/37823>). I plan to address > comments and get it in when I'm back from OOO. This should take care of > everything for Date, minus your bullet point #4 about integrating it into > JdbcUtil.java > > The Date type is relatively straightforward. Time and DateTime may need > some more thought though, specifically around the level of precision we > want to support. We discussed strategy in a previous thread ( > https://lists.apache.org/thread/xw6myhcr60q2c6nd6jb6gv395k3z2lyn) and doc > (https://s.apache.org/beam-timestamp-strategy), and reached an > understanding that for most flexibility we should parameterize time types > by precision (e.g. DateTime(p), Timestamp(p), Time(p)). > > @Claude van der Merwe <[email protected]> made a start by > implementing the parameterized Timestamp type in Java #36705 > <https://github.com/apache/beam/pull/36705> (not yet portable). We can do > something similar for DateTime and Time. > > Just want to make sure we keep this in mind when implementing things, but > it's a good direction. I'm happy that we're giving more attention to > portable types. > > Best, > Ahmed > > On Mon, Mar 16, 2026 at 4:36 PM Austin Bennett <[email protected]> wrote: > >> Hey folks, >> >> I'd like to propose making the existing Date, Time, and DateTime logical >> types standard portable types. This unblocks cross-language Date support — >> particularly Python SDK → IcebergIO (#37823). >> >> The Java SDK already defines these with java.time and portable URNs >> (beam:logical_type:date:v1, etc.), but they're missing from schema.proto's >> LogicalTypes.Enum and SchemaTranslation.STANDARD_LOGICAL_TYPES, so they >> degrade to UnknownLogicalType across the language boundary. >> >> Proposed changes: >> >> 1. Add DATE/TIME/DATETIME to LogicalTypes.Enum in schema.proto >> 2. Register them in STANDARD_LOGICAL_TYPES (SchemaTranslation.java) >> 3. Add Python SDK logical type classes (datetime.date, datetime.time) >> 4. Update JdbcUtil.java to also recognize the portable URNs (currently it >> only handles its own legacy "DATE"/"TIME" identifiers) >> >> I'm aware that JdbcIO currently uses its own non-portable logical types >> for dates and times, and Python has a temporary JdbcDateType/JdbcTimeType >> shim to match. This change would let us start deprecating those in favor of >> the standard portable types, moving toward #28359. >> >> All additive, and follows the MicrosInstant/FixedBytes pattern. Happy to >> put up the PRs [ first one here: >> https://github.com/apache/beam/pull/37870 ] — wanted to flag it since it >> touches the portable schema proto. OK if we don't like this direction. >> Please advise. >> >> Related: #37823, #25946, #28359 >> Thanks! >> Austin >> >>
