[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861869#comment-15861869
]
Egor Pahomov commented on SPARK-19524:
--
[~sowen], probably yes. I don't know. "Should process only
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861822#comment-15861822
]
Sean Owen commented on SPARK-19524:
---
Ah, so you _don't_ want to only read new files. The behavior of
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861600#comment-15861600
]
Egor Pahomov commented on SPARK-19524:
--
[~sowen],
Folder, which connect my streaming to:
{code}
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861319#comment-15861319
]
Sean Owen commented on SPARK-19524:
---
It's based on modification time by the way. Are the files' mod
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860023#comment-15860023
]
Egor Pahomov commented on SPARK-19524:
--
I'm really confused. I expected "new" to be the files
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859303#comment-15859303
]
Sean Owen commented on SPARK-19524:
---
Well, that is its definition of 'new' -- what do you expect
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858912#comment-15858912
]
Egor Pahomov commented on SPARK-19524:
--
[~uncleGen], sorry, I haven't understood.
> newFilesOnly
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858910#comment-15858910
]
Egor Pahomov commented on SPARK-19524:
--
[~srowen], based on documentation, which says "newFilesOnly
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858819#comment-15858819
]
Genmao Yu commented on SPARK-19524:
---
Current implementation will clear the old time-to-files mappings
[
https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858667#comment-15858667
]
Sean Owen commented on SPARK-19524:
---
I don't understand what you're reporting. Summarize expected,
10 matches
Mail list logo
