Hi,
Would anyone like to review this pull request:
https://github.com/apache/incubator-apex-malhar/pull/279
This is based on the design discussed in previous mails. I am open to
discuss it again if anyone is interested.
-Priyanka
On Mon, Apr 4, 2016 at 10:47 PM, Priyanka Gugale
wrote:
> Okay s
Okay so I will open the pull request soon.
-Priyanka
On Wed, Mar 23, 2016 at 1:35 PM, Yogi Devendra wrote:
> This looks OK. Let us build it incrementally.
>
> ~ Yogi
>
> On 23 March 2016 at 13:24, Sandeep Deshmukh
> wrote:
>
> > I would suggest that we go ahead with design as suggested by Priy
This looks OK. Let us build it incrementally.
~ Yogi
On 23 March 2016 at 13:24, Sandeep Deshmukh wrote:
> I would suggest that we go ahead with design as suggested by Priyanka where
> we have bandwidth setup for each operator separately. We can later extend
> this for bandwidth to be shared wit
I would suggest that we go ahead with design as suggested by Priyanka where
we have bandwidth setup for each operator separately. We can later extend
this for bandwidth to be shared with different input operators or for the
DAG as a whole.
Regards,
Sandeep
On Wed, Mar 23, 2016 at 11:51 AM, Priyan
Right now it's not for output operator, but one can very well use bandwidth
manager to keep track of bandwidth usage and limit your output speed. The
bigger challenge there would be, you won't be able to process window data
sent by upstream operator in same window. For that you need to do more than
That's a good question Chaitanya, Right now the bandwidth control is at
Input Operator level and not application level. So if you have two input
operator you need to set bandwidth on both separately by this design.
May be it would be good to have bandwidth control at Application level than
operator
Priyanka,
>From the design description it is not clear how it will be used to control
output bandwidth (point #2,3,4 mentioned by Sandeep)
~ Yogi
On 23 March 2016 at 11:39, Chaitanya Chebolu
wrote:
> This is very useful feature.
> I would like to know, how you are distributing the bandwidth fo
This is very useful feature.
I would like to know, how you are distributing the bandwidth for the below
situation:
- Two input operators say i1 and i2 are deployed on same node and both the
operators have bandwidthManager as plugin.
On Fri, Mar 18, 2016 at 5:43 PM, Priyanka Gugale
wrote:
> Hi,
>
Hi,
Thanks for inputs Sandeep, would take care of those points.
Here is high level design we are considering, We would have following
components:
*1.* *BandwidthManager*
This keeps track of current bandwidth usage of system and takes decision if
requested data bandwidth can be used right away or
The main purpose is not to handle back pressure but to limit bandwidth
usage by applications. This is useful in ingestion use cases. Typically
user needs to ingest say up to 1GB per sec and not more. The tuple size
may vary based on messages based tuples (few KBs) or block tuples for files
(few MB
Not sure if this is helpful, but there is already a utility in Malhar for
converting tuples per second to tuples per window. This allows the user to
define a property in tuples per second, then the operator can convert that
to tuples per window so it emits the correct number of tuples per window.
Hi Priyanka,
Indeed this is a useful feature.
I believe number bytes consumed per sec can as well translate to number of
tuples consumed per sec.
If above is correct, won't back pressure that is handled by bufferserver
help in your use case?
Thanks,
Chinmay.
On 2 Mar 2016 4:49 p.m., "Priyanka G
Many times we need to put bandwidth restrictions or put some limit on input
operator for number of bytes to be consumed per second. As I understand in
Apex there is no direct support for this feature.
I am planning to write a bandwidth manager which will help in limiting
bandwidth at Input operato
13 matches
Mail list logo
