stevenzwu commented on a change in pull request #2984:
URL: https://github.com/apache/iceberg/pull/2984#discussion_r690064162
##########
File path:
flink/src/main/java/org/apache/iceberg/flink/source/FlinkInputFormat.java
##########
@@ -72,12 +62,7 @@ public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) {
@Override
public FlinkInputSplit[] createInputSplits(int minNumSplits) throws
IOException {
- // Called in Job manager, so it is OK to load table from catalog.
- tableLoader.open();
Review comment:
> What is the lifecycle of FlinkInputFormat? Is it alright to pass a
read-only table to it that won't pick up any new snapshots or table properties?
@aokolnychyi I think it is ok to pass a read-only table to
FlinkInputFormat. it is used in two scenarios
1) batch mode: split discovery is done once at startup
2) streaming mode: it is used only in reader. dynamic/periodical split
discovery is done by `StreamingMonitorFunction`, which would require a
TableLoader to refresh table state
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
