[
https://issues.apache.org/jira/browse/SAMZA-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajo Thomas updated SAMZA-2741:
------------------------------
Description: Samza users running long running pipelines need a way to
gracefully drain them in preparation for an upgrade. The upgrade could be due
to changes to transforms or backward incompatible changes in data schema. The
only option currently available to the user is to abruptly stop the pipeline
and reset intermediate data which could lead to loss of buffered data. User
logic is expected to constantly evolve and schema changes are often backward
incompatible. It is therefore imperative to provide the ability to seamlessly
stop and upgrade their pipelines without losing any buffered data.
> Pipeline Drain: Support the ability to drain pipelines to allow incompatible
> intermediate schema changes
> --------------------------------------------------------------------------------------------------------
>
> Key: SAMZA-2741
> URL: https://issues.apache.org/jira/browse/SAMZA-2741
> Project: Samza
> Issue Type: New Feature
> Reporter: Ajo Thomas
> Assignee: Ajo Thomas
> Priority: Major
>
> Samza users running long running pipelines need a way to gracefully drain
> them in preparation for an upgrade. The upgrade could be due to changes to
> transforms or backward incompatible changes in data schema. The only option
> currently available to the user is to abruptly stop the pipeline and reset
> intermediate data which could lead to loss of buffered data. User logic is
> expected to constantly evolve and schema changes are often backward
> incompatible. It is therefore imperative to provide the ability to seamlessly
> stop and upgrade their pipelines without losing any buffered data.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
