[
https://issues.apache.org/jira/browse/APEXMALHAR-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226158#comment-15226158
]
Priyanka Gugale commented on APEXMALHAR-2045:
---------------------------------------------
high level design we are considering, We would have following components:
1. BandwidthManager
This keeps track of current bandwidth usage of system and takes decision if
requested data bandwidth can be used right away or not. To do this it used
Leaky bucket algorithm where it emits data as long as it has not overused
bandwidth (i.e. bandwidth consumption is >=0) and then wait to accumulate
bandwidth for a while (till bandwidth goes from -ve value to +ve).
2. BandwidthLimitingInputOperator
Any Input operator which want to implement bandwidth restriction should
implement BandwidthLimitingInputOperator. The operator have abstract method to
initialize instance of BandwidthManager and a method to emit tuple with
bandwidth restriction to emit tuples as per available bandwidth.
3. BandwidthPartitioner
Bandwidth partitioner is introduced for static partitioning. If static
partitioning is used by default StatelessPartitioner class is initialized. With
bandwidth restriction we want to equally divide bandwidth amongst available
partitions. BandwidthPartitioner should take care of it. It extends
StatelessPartitioner, it just sets right bandwidth on all partitions after
StatelessPartitioner creates/deletes partitiolns. In case of dynamic
partitioning the operator implementing definePartitions, should take care of
bandwidth distribution.
This design takes care of basic bandwidth restriction, also takes care of
partitions by equally distributing available bandwidth among all partitions.
Also this is open enough to do further modifications to take care of complex
situations.
> Add bandwidth control feature to Apex
> -------------------------------------
>
> Key: APEXMALHAR-2045
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2045
> Project: Apache Apex Malhar
> Issue Type: Task
> Reporter: Priyanka Gugale
> Assignee: Priyanka Gugale
>
> bandwidth restrictions on input operator for number of bytes to be consumed
> per second.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
