GitHub user dnishimura opened a pull request:
https://github.com/apache/samza/pull/860
SAMZA-1985: Startpoint and StartpointManager implementation.
This is the first PR for
[SEP-18](https://cwiki.apache.org/confluence/display/SAMZA/SEP-18%3A+Startpoints+-+Manipulating+Starting+Offsets+for+Input+Streams).
Please see updated SEP-18 for details.
This PR implements the StartpointManager and Startpoint model and the
initial integration with the OffsetManager. The OffsetManager manages the
deletion of Startpoints when the initial checkpoint commits happen per task
after start-up.
The immediate follow-ons to this PR are:
1. Have the various `JobCoordinators` to re-map the Startpoints
appropriately to each task by utilizing the
`StartpointManager#groupStartpointsPerTask(SystemStreamPartition,
SystemStreamPartitionGrouper)` method implemented in this PR. SEP-18 describes
this in more detail.
2. Add a new `SystemConsumer#register(SystemStreamPartition, Startpoint)`
interface and implement the interface for each provided system
(`KafkaSystemConsumer`, `EventHubSystemConsumer`, etc...) and integrate the new
interface appropriately.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dnishimura/samza samza-1985-startpoint-manager
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/860.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #860
----
commit b8437e233f6d13c4ddbdc3cbe63d540d459611fe
Author: Daniel Nishimura <dnishimura@...>
Date: 2018-12-12T22:49:51Z
SAMZA-1985: Startpoint and StartpointManager implementation.
----
---
