vamsikarnika commented on code in PR #639:
URL: https://github.com/apache/incubator-xtable/pull/639#discussion_r1942750445
##########
xtable-utilities/src/main/java/org/apache/xtable/utilities/RunCatalogSync.java:
##########
@@ -316,6 +330,8 @@ public static class SourceTableIdentifier {
* not strictly registered in a catalog, as long as the format and
location are known
*/
StorageIdentifier storageIdentifier;
+ /** Specifies properties of source table. */
+ Map<String, String> tableProperties;
Review Comment:
When source or target table format is HUDI, we need partition spec of the
table for building the HudiConversionSourceProvider. we read partition spec
config(`xtable.source.partition_field_spec_config`) from table's additional
properties which are extracted from catalog table properties. If the catalog
table doesn't contain this config, catalog sync fails. So adding this extra
tableProperties config to provide any extra configs that might be missing in
catalog table properties.
```
datasets:
- sourceCatalogTableIdentifier:
tableIdentifier:
hierarchicalId: "target_iceberg_hms_db_1.target_iceberg_hms_tb_1"
storageIdentifier:
tableBasePath: <basepath>
tableDataPath: <datapath>
tableName: trips_table
partitionSpec: id:VALUE
tableFormat: "ICEBERG"
```
```
datasets:
- sourceCatalogTableIdentifier:
tableIdentifier:
hierarchicalId: "target_glue_delta_db_1.target_delta_tb_1"
tableProperties:
"xtable.hudi.source.partition_field_spec_config": "id:VALUE"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
