[
https://issues.apache.org/jira/browse/APEXCORE-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390152#comment-15390152
]
ASF GitHub Bot commented on APEXCORE-482:
-----------------------------------------
Github user vrozov commented on the issue:
https://github.com/apache/apex-core/pull/358
Most of the time default unifier (the one that passes tuples unmodified
from input streams to an output stream) is used. For that case, it is necessary
to benchmark the change (please see my previous TODO comment). In general,
there should be a way for an application designer to specify an unifier stream
locality for an output port (thread, container, rack or no locality) with some
reasonable defaults (whether it is CONTAINER_LOCAL or THREAD_LOCAL). It should
be also possible to set not only stream locality for the output port, but also
buffer server memory size or queue size. Currently, buffer server queue size
when unifier is deployed into a separate container (Nx1 partition) is the same
as the original operator buffer server memory. This limitation seems to come
from using the same StreamMeta both for the unifier output port and for the
partitioned operator output port.
> Default Unifiers should be ThreadLocal with downstream operators
> ----------------------------------------------------------------
>
> Key: APEXCORE-482
> URL: https://issues.apache.org/jira/browse/APEXCORE-482
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Sandesh
> Assignee: Sandesh
>
> Default Unifiers are currently deployed ( except in Mx1 ) as CONTAINER_LOCAL
> with the downstream operator. It is better to make it THREAD_LOCAL to improve
> the performance.
> Custom unifiers can still be CONTAINER_LOCAL and possibly allow the users to
> control the behaviours.
> Note: Created a prototype to understand the root cause :
> https://github.com/sandeshh/apex-core/commit/550f31c770b704360c813924e85a9fffeaec7c53
> During PhysicalPlan creation downstream operator's input port won't have
> stream locality assigned. So StreamingContainerAgent assigns CONTAINER_LOCAL
> Actual implementation should be done in Physical Plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
