[ https://issues.apache.org/jira/browse/HUDI-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xiaozhongcheng closed HUDI-4603. -------------------------------- Fix Version/s: (was: 0.13.0) Resolution: Fixed > Improve HMS Catalog function in flink > ------------------------------------- > > Key: HUDI-4603 > URL: https://issues.apache.org/jira/browse/HUDI-4603 > Project: Apache Hudi > Issue Type: Improvement > Reporter: xiaozhongcheng > Priority: Major > Labels: pull-request-available > > 1. Make users can choose to sync ro and rt table or not when using hms > catalog in flink. > Currently, if users use hms catalog in flink, ro and rt table will be also > sync to the hms. > But If I just want to sync the metadata of hudi table, I don't want to sync > ro and rt table. > So, I think users should be able to choose to sync the ro and rt table or not. > > 2. Make users can sync ro and rt table correctly when the table is > partitioned table. > Currently, If users create the partitioned table in hms catalog, but the > partition field is not in the form of yyyy/mm/dd. If they don't make > hive_sync.partition_extractor_class to > org.apache.hudi.hive.HiveStylePartitionValueExtractor, ro and rt table will > not be synced correctly > The stacks is list below: > > {code:java} > org.apache.hudi.exception.HoodieException: Got runtime exception when hive > syncing student22 > at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:144) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sink.StreamWriteOperatorCoordinator.doSyncHive(StreamWriteOperatorCoordinator.java:335) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sink.StreamWriteOperatorCoordinator.syncHive(StreamWriteOperatorCoordinator.java:326) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sink.StreamWriteOperatorCoordinator.handleEndInputEvent(StreamWriteOperatorCoordinator.java:426) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$handleEventFromOperator$3(StreamWriteOperatorCoordinator.java:278) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:1.8.0_241] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_241] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_241] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_241] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_241] > Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed to sync > partitions for table student22_ro > at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:340) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:232) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:157) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:141) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > ... 10 more > Caused by: java.lang.IllegalArgumentException: Partition path school=beida is > not in the form yyyy/mm/dd > at > org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor.extractPartitionValuesInPath(SlashEncodedDayPartitionValueExtractor.java:58) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at > org.apache.hudi.sync.common.HoodieSyncClient.getPartitionEvents(HoodieSyncClient.java:144) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:318) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:232) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:157) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:141) > ~[hudi-flink1.15-bundle-0.12.0-rc2.jar:0.12.0-rc2] > ... 10 more {code} > > > > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)