One of use cases is a batch application with multiple reduce/aggregate operators.
Thank you, Vlad Отправлено с iPhone > On Apr 1, 2017, at 10:23, Munagala Ramanath <r...@datatorrent.com> wrote: > > What's a use case for this ? > > Ram > >> On Sat, Apr 1, 2017 at 8:12 AM, Vlad Rozov <v.ro...@datatorrent.com> wrote: >> >> All, >> >> Currently Apex assumes that an operator can emit on any defined output >> port and all streams defined by a DAG are active. I'd like to propose an >> ability for an operator to open and close output ports. By default all >> ports defined by an operator will be open. In the case an operator for any >> reason decides that it will not emit tuples on the output port, it may >> close it. This will make the stream inactive and the application master may >> undeploy the downstream (for that input stream) operators. If this leads to >> containers that don't have any active operators, those containers may be >> undeployed as well leading to better cluster resource utilization and >> better Apex elasticity. Later, the operator may be in a state where it >> needs to emit tuples on the closed port. In this case, it needs to re-open >> the port and wait till the stream becomes active again before emitting >> tuples on that port. Making inactive stream active again, requires the >> application master to re-allocate containers and re-deploy the downstream >> operators. >> >> It should be also possible for an application designer to mark streams as >> inactive when an application starts. This will allow the application master >> avoid reserving all containers when the application starts. Later, the port >> can be open and inactive stream become active. >> >> Thank you, >> >> Vlad >> >> > > > -- > > _______________________________________________________ > > Munagala V. Ramanath > > Software Engineer > > E: r...@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam > > www.datatorrent.com | apex.apache.org
