Github user mxm commented on a diff in the pull request:
https://github.com/apache/flink/pull/2618#discussion_r83020529
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java
---
@@ -60,26 +63,27 @@
private static final Logger LOG =
LoggerFactory.getLogger(ContinuousFileMonitoringFunction.class);
/**
- * The minimum interval allowed between consecutive path scans. This is
applicable if the
- * {@code watchType} is set to {@code PROCESS_CONTINUOUSLY}.
+ * The minimum interval allowed between consecutive path scans.
+ * <p><b>NOTE:</b> Only applicable to the {@code PROCESS_CONTINUOUSLY}
mode.
*/
- public static final long MIN_MONITORING_INTERVAL = 100l;
+ public static final long MIN_MONITORING_INTERVAL = 1l;
/** The path to monitor. */
private final String path;
- /** The default parallelism for the job, as this is going to be the
parallelism of the downstream readers. */
+ /** The parallelism of the downstream readers. */
private final int readerParallelism;
/** The {@link FileInputFormat} to be read. */
private FileInputFormat<OUT> format;
- /** How often to monitor the state of the directory for new data. */
+ /** The interval between consecutive path scans. */
private final long interval;
/** Which new data to process (see {@link FileProcessingMode}. */
private final FileProcessingMode watchType;
+ /** The maximum file modification time seen so far. */
private Long globalModificationTime;
--- End diff --
I wonder, should this be `volatile` because it is accessed by the
checkpoint thread?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
