GitHub user ConcurrencyPractitioner opened a pull request:
https://github.com/apache/spark/pull/21651
[SPARK-18258] Sink need access to offset representation
## What changes were proposed in this pull request?
Currently, sinks only have access to the batchId and the data, not the
actual offset representation.
The goal of this PR is to expose this representation to sinks via
```addBatch```.
## How was this patch tested?
Existing unit tests (needs to be changed to also test for offsetSeqs)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ConcurrencyPractitioner/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21651.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21651
----
commit f27be2d323fb287876032b59dd078fc65e9b180d
Author: Richard Yu <yohan.richard.yu@...>
Date: 2018-06-28T00:44:34Z
[SPARK-18258] Init Commit
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
