[GitHub] [hudi] danny0405 commented on a diff in pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…
danny0405 commented on code in PR #9122:
URL: https://github.com/apache/hudi/pull/9122#discussion_r1252517410
##
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##
@@ -144,18 +144,7 @@ public BaseHoodieTableFileIndex(HoodieEngineContext
engineContext,
this.engineContext = engineContext;
this.fileStatusCache = fileStatusCache;
-// The `shouldListLazily` variable controls how we initialize the
TableFileIndex:
-// - non-lazy/eager listing (shouldListLazily=false): all partitions and
file slices will be loaded eagerly during initialization.
-// - lazy listing (shouldListLazily=true): partitions listing will be
done lazily with the knowledge from query predicate on partition
-//columns. And file slices fetching only happens for partitions
satisfying the given filter.
-//
-// In SparkSQL, `shouldListLazily` is controlled by option
`REFRESH_PARTITION_AND_FILES_IN_INITIALIZATION`.
-// In lazy listing case, if no predicate on partition is provided, all
partitions will still be loaded.
-if (shouldListLazily) {
- this.tableMetadata = createMetadataTable(engineContext, metadataConfig,
basePath);
Review Comment:
Ignore, it is created in `doRefresh`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…
danny0405 commented on code in PR #9122:
URL: https://github.com/apache/hudi/pull/9122#discussion_r1252517112
##
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##
@@ -144,18 +144,7 @@ public BaseHoodieTableFileIndex(HoodieEngineContext
engineContext,
this.engineContext = engineContext;
this.fileStatusCache = fileStatusCache;
-// The `shouldListLazily` variable controls how we initialize the
TableFileIndex:
-// - non-lazy/eager listing (shouldListLazily=false): all partitions and
file slices will be loaded eagerly during initialization.
-// - lazy listing (shouldListLazily=true): partitions listing will be
done lazily with the knowledge from query predicate on partition
-//columns. And file slices fetching only happens for partitions
satisfying the given filter.
-//
-// In SparkSQL, `shouldListLazily` is controlled by option
`REFRESH_PARTITION_AND_FILES_IN_INITIALIZATION`.
-// In lazy listing case, if no predicate on partition is provided, all
partitions will still be loaded.
-if (shouldListLazily) {
- this.tableMetadata = createMetadataTable(engineContext, metadataConfig,
basePath);
Review Comment:
The initialization of `tableMetadata` is removed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
