nsivabalan commented on code in PR #18384:
URL: https://github.com/apache/hudi/pull/18384#discussion_r3277598468
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieTableServiceClient.java:
##########
@@ -795,6 +795,11 @@ protected HoodieTable
createTableAndValidate(HoodieWriteConfig config,
if (!skipValidation) {
CommonClientUtils.validateTableVersion(table.getMetaClient().getTableConfig(),
config);
}
+ // Overlay the persisted POPULATE_META_FIELDS and META_FIELDS_EXCLUDE_LIST
onto the
+ // shared writer config so table services (clustering/compaction/clean)
see the on-disk
+ // state. Safe to mutate in place: these properties are immutable for the
lifetime of
+ // the table. Mirrors the same step in
BaseHoodieWriteClient.createTableAndValidate.
+ table.getMetaClient().getTableConfig().overlayMetaFieldProps(config);
Review Comment:
but there are quite a few call sites (writer factories), where we do not
send tableConfig, but just the writeConfig. and hence went ahead w/ this
change.
and wrt mutability, any table property is immutable. So, once we start a
pipeline, the properties fetched from table config can never change.
for the meta field exclude list, a. either during upgrade. b. via hudi cli.
in other cases, writes will fail if someone tries to change the table
property.
I can raise a separate diff to showcase how much of a change we are looking
at if we wanted to take a look to see, how much of a change is, wiring table
config to all call sites.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
