[
https://issues.apache.org/jira/browse/APEXCORE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise updated APEXCORE-163:
----------------------------------
Labels: (was: roadmap)
> Dynamic application property changes
> ------------------------------------
>
> Key: APEXCORE-163
> URL: https://issues.apache.org/jira/browse/APEXCORE-163
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Pramod Immaneni
> Assignee: Pramod Immaneni
>
> Apex support modification of operator properties at runtime but the current
> implemenations has the following shortcomings.
> 1. Property is not set across all partitions on the same window as individual
> partitions can be on different windows when property change is initiated from
> client resulting in inconsistency of data for those windows. I am being
> generous using the word inconsistent.
> 2. Sometimes properties need to be set on more than one logical operators at
> the same time to achieve the change the user is seeking. Today they will be
> two separate changes happening on two different windows again resulting in
> inconsistent data for some windows. These would need to happen as a single
> transaction.
> 3. If there is an operator failure before a committed checkpoint after an
> operator property is dynamically changed the operator will restart with the
> old property and the change will not be re-applied.
> Tim and myself did some brainstorming and we have a proposal to overcome
> these shortcomings. The main problem in all the above cases is that the
> property changes are happening out-of-band of data flow and hence independent
> of windowing. The proposal is to bring the property change request into the
> in-band dataflow so that they are handled consistently with windowing and
> handled distributively.
> The idea is to inject a special property change tuple containing the property
> changes and the identification information of the operator's they affect into
> the dataflow at the input operator. The tuple will be injected at window
> boundary after end window and before begin window and as this tuple flows
> through the DAG the intended operators properties will be modifed. They will
> all be modified consistently at the same window. The tuple can contain more
> than one property changes for more than one logical operators and the change
> will be applied consistently to the different logical operators at the same
> window. In case of failure the replay of tuples will ensure that the property
> change gets reapplied at the correct window.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
