That's one option, though stats listener may provide better context for decision making.
This is needed for better batch support. -- sent from mobile On Feb 23, 2016 8:36 PM, "Sandesh Hegde" <[email protected]> wrote: > Experimental, Can we use iteration to send the feedback and based on that > adjust the bandwidth? > > On Tue, Feb 23, 2016 at 8:32 PM Thomas Weise <[email protected]> > wrote: > >> Priyanka, that's right. In this case we want to process as fast as >> possible without overwhelming the downstream operators. >> >> -- >> sent from mobile >> On Feb 23, 2016 8:25 PM, "Priyanka Gugale" <[email protected]> >> wrote: >> >>> Hi, >>> >>> I have seen that code in Ingestion, and as I remember it had some issues >>> and wasn't working. >>> I don't have exactly what Thomas asked, but we can use Bandwidth limit >>> feature, where splitter itself emits based on configured limit, but yes it >>> isn't based on load on readers. >>> >>> -Priyanka >>> >>> On Wed, Feb 24, 2016 at 5:33 AM, Chandni Singh <[email protected]> >>> wrote: >>> >>>> Hi Thomas, >>>> >>>> I had written a StatsListener which when set on FileSplitter and >>>> BlockReader would automatically throttle block emission rate. The stats >>>> listener will send a setProperty command to FileSplitter however if I >>>> recall correctly, this changed the property only on the logical operator >>>> and wasn't propagated to the physical operator. >>>> >>>> It was a while back so I am not sure if that was still the issue or >>>> there was something else wrong my implementation. >>>> >>>> Thanks, >>>> Chandni >>>> >>>> On Tue, Feb 23, 2016 at 3:48 PM, Thomas Weise <[email protected]> >>>> wrote: >>>> >>>>> Is there any support for the splitter to automatically throttle block >>>>> emission rate based on the readers throughput? >>>>> >>>>> -- >>>>> sent from mobile >>>>> >>>> >>>> >>>
