You're hitting an existing issue
https://issues.apache.org/jira/browse/SPARK-17604. While there's no active
PR to address it, I've been planning to take a look sooner than later.
Btw, you may also want to take a look at my previous mail - the topic on
the mail thread was regarding file stream
Hi Team,
While Running Spark Below are some finding.
1. FileStreamSourceLog is responsible for maintaining input source file
list.
2. Spark Streaming delete expired log files on the basis of s
*park.sql.streaming.fileSource.log.deletion* and
