[GitHub] spark pull request #20683: [SPARK-8605] Exclude files in StreamingContext. t...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20683


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20683: [SPARK-8605] Exclude files in StreamingContext. t...

[ConcurrencyPractitioner]

GitHub user ConcurrencyPractitioner opened a pull request:

https://github.com/apache/spark/pull/20683

[SPARK-8605] Exclude files in StreamingContext. textFileStream(direct…

…ory)

## What changes were proposed in this pull request?

In this PR, a extra boolean expression was added to test if a regex was 
present. If returned true, then we exclude the file.

## How was this patch tested?

No tests were added. 


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ConcurrencyPractitioner/spark master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20683.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20683


commit 4ad0fe1dc1513a5feba963f902742fecc714e4d0
Author: Richard Yu 
Date:   2018-02-27T04:05:37Z

[SPARK-8605] Exclude files in StreamingContext. textFileStream(directory)




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org