GitHub user deepak-narkhede opened a pull request:
https://github.com/apache/apex-core/pull/466
APEXCORE-634 Apex Platform unable to set unifier attributes for modules
Problem: Unable to set Unifier attributes of output port within modules
Description: When modules are flatten in logical Plan of DAG, only top
level attributes are cloned of OperatorMeta. Unifier attributes are not copied
in PortMapping for output ports.
Solution: Clone the unifier attributes while flattening DAG in logical
Plan.
Testing done for custom application with and without modules also with app
template HDFS to S3 module. Have had debug logs and stack traces to verify the
fix.
One of the snip of logs had set the unifier attribute TIMEOUT_WINDOW_COUNT
to 10 ( i.e 1000 millisec)
**Without fix:**
**<log_snip>**
2017-02-06 15:51:31,539 WARN
com.datatorrent.stram.StreamingContainerManager: UNIFIER operator
PTOperator[id=4,name=**genmodule$gen.out#unifier]** committed window
ffffffffffffffff, recovery window ffffffffffffffff, current time 1486376491539,
last window id change time 0, **window processing timeout millis 60000**
**</log_snip>**
**With Fix:**
**<log_snip>**
2017-02-06 14:22:49,602 WARN
com.datatorrent.stram.StreamingContainerManager: UNIFIER operator
PTOperator[id=4,name=**genmodule$gen.out#unifier]** committed window
ffffffffffffffff, recovery window ffffffffffffffff, current time 1486371169602,
last window id change time 0, **window processing timeout millis 1000**
**</log_snip>**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/deepak-narkhede/apex-core APEXCORE-634
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-core/pull/466.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #466
----
commit 3ab8916058e3d5ccaecb9dc84c9a96d6ab22ebec
Author: deepak-narkhede <[email protected]>
Date: 2017-02-07T11:09:07Z
APEXCORE-634 Apex Platform unable to set unifier attributes for modules in
DAG.
Problem: Unable to set Unifier attributes of output port within modules
Description: When modules are flatten in logical Plan of DAG, only top
level attributes are cloned of OperatorMeta.
Unifier attributes are not copied in PortMapping for output ports.
Solution: Clone the unifier attributes while flattening DAG in logical
Plan.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
