Vlad Rozov created APEXCORE-641:
-----------------------------------
Summary: Subscribers/DataListeners may not be scheduled to execute
even when they have data to process
Key: APEXCORE-641
URL: https://issues.apache.org/jira/browse/APEXCORE-641
Project: Apache Apex Core
Issue Type: Bug
Components: Buffer Server
Affects Versions: 3.5.0, 3.4.0, 3.2.1, 3.3.0, 3.2.0, 3.6.0
Reporter: Vlad Rozov
Assignee: Vlad Rozov
Buffer server iterates over DataListeners aka LogicalNodes and each LogicalNode
tries to send to it's downstream all data that Publisher added to the DataList.
When an output port is connected to multiple partitions or downstream operators
(2 or more DataListeners/LogicalNodes) there may be more data published to the
DataList after first few DataListeners in the listeners set iterated over
DataList and reached the last block published so far. The data published while
the last DataListeners sends data to it's downstream will not be processed by
other DataListeners until Publisher adds more data to the DataList. This may
lead to blocked operators, as Buffer server may stop processing data completely
in case Publisher fills more than one Data block while a single DataListener
sends data to it's downstream and there are more Subscribers/DataListeners than
number of in memory blocks allowed (8). In such case, Publisher will be
suspended, and there will be no task scheduled to process data already
published to the DataList.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
