[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2019-02-19 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772716#comment-16772716 ] Jungtaek Lim commented on SPARK-20971: -- I'm working on reducing actual file entities, which are

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2019-02-19 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772632#comment-16772632 ] Jungtaek Lim commented on SPARK-20971: -- Maybe better to clarify what we would like to do here. *

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2019-02-19 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772351#comment-16772351 ] Jungtaek Lim commented on SPARK-20971: -- I guess this can be also handled in SPARK-20568 since

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2018-09-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610639#comment-16610639 ] Wenchen Fan commented on SPARK-20971: - I'm removing the target version, since we are not going to

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061596#comment-16061596 ] Apache Spark commented on SPARK-20971: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043150#comment-16043150 ] Shixiong Zhu commented on SPARK-20971: -- FileStreamSource saves the seen files in the disk/HDFS, we

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-08 Thread Fei Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042427#comment-16042427 ] Fei Shao commented on SPARK-20971: -- Hi Zhu, What does metadata logs stands for please? > Purge the