[GitHub] spark issue #21651: [SPARK-18258] Sink need access to offset representation

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/21651 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21651: [SPARK-18258] Sink need access to offset representation

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/21651 I am uncertain about some of the ways we should transfer the data stored in OffsetSeqs to external storage (e.g. like KafkaSink which I mentioned before

[GitHub] spark issue #21651: [SPARK-18258] Sink need access to offset representation

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/21651 cc @koeninger --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21651: [SPARK-18258] Sink need access to offset represen...

GitHub user ConcurrencyPractitioner opened a pull request: https://github.com/apache/spark/pull/21651 [SPARK-18258] Sink need access to offset representation ## What changes were proposed in this pull request? Currently, sinks only have access to the batchId and the data

[GitHub] spark issue #21124: [SPARK-23004][SS] Ensure StateStore.commit is called onl...

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/21124 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/20683 @jerryshao In Spark Streaming, I think ```.tmp``` is used as a suffix to indicate that the object was a file, although I do not know if this is universal

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/20683 Jenkins test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20683: [SPARK-8605] Exclude files in StreamingContext. t...

GitHub user ConcurrencyPractitioner opened a pull request: https://github.com/apache/spark/pull/20683 [SPARK-8605] Exclude files in StreamingContext. textFileStream(direct… …ory) ## What changes were proposed in this pull request? In this PR, a extra boolean