danny0405 commented on code in PR #19205:
URL: https://github.com/apache/hudi/pull/19205#discussion_r3651808817
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -124,7 +125,11 @@ private Option<HoodieSchema>
getTableSchemaFromDataFileInternal() {
* @throws Exception
*/
public HoodieSchema getTableSchema() throws Exception {
- return getTableSchema(metaClient.getTableConfig().populateMetaFields());
+ // Include meta fields whenever the table's meta-fields mode populates any
of them. Under
+ // selective modes (COMMIT_TIME_ONLY / FILE_NAME_ONLY /
COMMIT_TIME_AND_FILE_NAME) the meta
+ // columns exist as physical nullable Parquet columns even though
populateMetaFields() is false,
+ // and read paths (e.g. incremental relations) must see them in the
projected schema.
+ return getTableSchema(metaClient.getTableConfig().getMetaFieldsMode() !=
MetaFieldsMode.NONE);
Review Comment:
so for selective modes, all the metadata fields are still in the shema, is
it for compatibility purposes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
