[
https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471264#comment-17471264
]
Harsha Teja Kanna edited comment on HUDI-2909 at 1/9/22, 2:05 AM:
------------------------------------------------------------------
I am not able to determine if I fall under user type c or a/b :) from the
Github issue or the above description.
Can you please help understand if I have to recreate the dataset?
was (Author: h7kanna):
I am not able to determine if I fall under user type c or a/b :) from the
Github issue or the above description.
I can you please help understand if I have to recreate the dataset?
> Partition field parsing fails due to KeyGenerator giving inconsistent value
> for logical timestamp type
> ------------------------------------------------------------------------------------------------------
>
> Key: HUDI-2909
> URL: https://issues.apache.org/jira/browse/HUDI-2909
> Project: Apache Hudi
> Issue Type: Bug
> Components: DeltaStreamer
> Reporter: Harsha Teja Kanna
> Assignee: Sagar Sumit
> Priority: Blocker
> Labels: core-flow-ds, pull-request-available, sev:critical
> Fix For: 0.10.1
>
>
> Existing table has timebased keygen config show below
> hoodie.deltastreamer.keygen.timebased.timestamp.type=SCALAR
> hoodie.deltastreamer.keygen.timebased.output.timezone=GMT
> hoodie.deltastreamer.keygen.timebased.output.dateformat=yyyy/MM/dd
> hoodie.deltastreamer.keygen.timebased.timestamp.scalar.time.unit=MICROSECONDS
> hoodie.deltastreamer.keygen.timebased.input.timezone=GMT
> hoodie.datasource.write.partitionpath.field=lastdate:timestamp
> hoodie.datasource.write.operation=upsert
> hoodie.deltastreamer.transformer.sql=SELECT session.id, session.rid,
> session.mid, to_timestamp(session.lastdate) as lastdate,
> to_timestamp(session.updatedate) as updatedate FROM <SRC> a
>
> Upgrading to 0.10.0 from 0.9.0 fails with exception
> org.apache.hudi.exception.HoodieKeyGeneratorException: Unable to parse input
> partition field :2021-12-01 10:13:34.702
> Caused by: org.apache.hudi.exception.HoodieNotSupportedException: Unexpected
> type for partition field: java.sql.Timestamp
> at
> org.apache.hudi.keygen.TimestampBasedAvroKeyGenerator.getPartitionPath(TimestampBasedAvroKeyGenerator.java:211)
> at
> org.apache.hudi.keygen.TimestampBasedAvroKeyGenerator.getPartitionPath(TimestampBasedAvroKeyGenerator.java:133)
> *Workaround fix:*
> Reverting this
> https://github.com/apache/hudi/pull/3944/files#diff-22fb52b5cf28727ba23cb8bd4be820432a4e396ce663ac472a4677e889b7491eR543
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
