+1 for this. ~ Chinmay.
On Mon, Dec 28, 2015 at 2:27 PM, Priyanka Gugale <[email protected]> wrote: > Hi, > > In Malhar we have an > operator AbstractBatchTransactionableStoreOutputOperator which creates > batches based on tuples received in a window. At the end of the window > these batches are sent to database for processing. > There is no way to configure MAX_SIZE on these batches. Based on input rate > the batch sizes can grow very high, and we might want to restrict batch > size. > > Any operator can extend and do batch management on their own, but I see it > as generic requirement and IMO we should change base class i.e. > AbstractBatchTransactionableStoreOutputOperator class to accept MAX_SIZE > for batch from outside. > > Any opinion on this? > > -Priyanka >
