[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-03-16 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20683 Don't really understand the issue itself. Which filesystem used this case? Why is it not possible to use Hadoop-compatible filesystem like HDFS for instance? This supports atomic rename. [See

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-03-11 Thread ConcurrencyPractitioner
Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/20683 @jerryshao In Spark Streaming, I think ```.tmp``` is used as a suffix to indicate that the object was a file, although I do not know if this is universal. ---

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-02-27 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20683 > a extra boolean expression was added to test if a regex was present. Can you please explain what's the meaning of "if a regex was present"? Seems the fix is not so necessary. If

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20683 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20683 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-02-26 Thread ConcurrencyPractitioner
Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/20683 Jenkins test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional