[
https://issues.apache.org/jira/browse/APEXCORE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968777#comment-15968777
]
Tushar Gosavi commented on APEXCORE-408:
----------------------------------------
I have updated the document, please go through it. The approach I am taking is
summarized below
- extend dag interface with following methods
- removeOperator/removeStream
- add startTransaction method on LogicalPlan which will return a DAG object,
which is a instance of TransactionalDAG which is clone of the dag with
reference to operator and ports from the original dag.
TransactionDAG maintains the reference to original dag and has a additional
commit method. On commit if the new DAG is valid then the same changes are
applied to the original DAG.
- user can change the cloned dag as per his requirement. Currently I am
providing the hooks to change the DAG through StatsListener by extending
StatsListener interface having additional context object. later the hooks could
also be provided through DAGExecutionPluginContext which has access to more
information from stram than statslistener.
- A Commit handler interface is added which commit method is call to notify
type of changes performed to the DAG. The PhysicalPlan will be changed through
this handler.
> Ability to schedule Sub-DAG from running application
> ----------------------------------------------------
>
> Key: APEXCORE-408
> URL: https://issues.apache.org/jira/browse/APEXCORE-408
> Project: Apache Apex Core
> Issue Type: Sub-task
> Reporter: Thomas Weise
> Assignee: Tushar Gosavi
>
> Today it is possible to add operators to a running application from the
> CLI/client. It should be possible to do this from within the application
> also. This can be used to expand/remove a Sub-DAG on demand, triggered by
> application specific logic. It will enable use cases such as batch
> applications that perform multiple stages of processing where not all
> resources are required at the same time or the logic of subsequent steps
> depends on the execution of previous steps.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
