hudi-bot opened a new issue, #14599: URL: https://github.com/apache/hudi/issues/14599
Currently bootstrap implementation is not able to handle partition columns correctly when the source data has *hive-style partitioning*, as is also mentioned in https://jira.apache.org/jira/browse/HUDI-915 The schema inferred while performing bootstrap and stored in the commit metadata does not have partition column schema(in case of hive partitioned data). As a result during hive-sync when hudi tries to determine the type of partition column from that schema, it would not find it and assume the default data type *string*. Here is where partition column schema is determined for hive-sync: [https://github.com/apache/hudi/blob/master/hudi-hive-sync/src/main/java/org/apache/hudi/hive/util/HiveSchemaUtil.java#L417] Thus no matter what the data type of partition column is in the source data (atleast what spark infers it as from the path), it will always be synced as string. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-992 - Type: Bug - Epic: https://issues.apache.org/jira/browse/HUDI-1265 - Affects version(s): - 0.9.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
