Bharath Kumarasubramanian created SAMZA-2791:
------------------------------------------------
Summary: Introduce callback timeout specific to watermark messages
Key: SAMZA-2791
URL: https://issues.apache.org/jira/browse/SAMZA-2791
Project: Samza
Issue Type: New Feature
Reporter: Bharath Kumarasubramanian
Assignee: Bharath Kumarasubramanian
Currently, watermark is implemented as a special message within Samza. However,
in terms of processing semantics, it shares similar behavior to normal messages
processed by the task. i.e., `task.callback.timeout.ms`, a configuration to
tune the time until which runloop waits for a message to be processed applies
to both watermark and normal messages.
However, this tie up constrains watermark processing logic to be bounded by the
processing messages time bound. For Beam on Samza, we use watermark as a
trigger to execute event timers which can take a long time depending on the
number of timers accumulated. Especially, when the application is down, the
timers accumulated could be too large and users will have to tune this
configuration which will also impact fault tolerance behavior in case of
failures/delays during processing messages.
*Changes*
Introduce callback timeout configuration specific to watermark
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
