bkonold opened a new pull request #1385:
URL: https://github.com/apache/samza/pull/1385
**Issues**: Side input consumption / processing currently happens in two
threads in `ContainerStorageManager`; one thread polling messages from
`SystemConsumers` and writing entries to the stores, and another thread
executing store flush.
Moving this processing to `RunLoop` will simplify further changes needed to
support transactional state in standby containers, as that requires ssp/task
level choose control and queuing of messages within a task, something that
`RunLoop` already does.
Other benefits of this change include making it easier in the future to
implement features like broadcast streams, side input priorities, side input
concurrency etc. It is also an incremental step toward addressing SAMZA-2494,
which may require side inputs to eventually be merged back into
`SamzaContainer`'s `RunLoop` instance in order for side input exceptions to be
propagated back to container.
**Changes**: Consolidates the two side input threads in
`ContainerStorageManager` by instead constructing instances of `SideInputTask`
which are executed by a single `RunLoop` instance/thread.
New side input processing metrics emitted by `SamzaContainerMetrics` and
`TaskInstanceMetrics` are given their own namespace in order to differentiate
from the primary container / run loop.
**Tests**: Existing tests are left in place.
**API Changes**: None.
**Upgrade Instructions**: None.
**Usage Instructions**: None.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
