Just to close this thread with numbers, this change is a part of 3.4 release.
Tested this change while doing Yahoo Benchmark, Throughput increased from ~650K tuples/sec to over 1,000K tuples/sec, over 50% improvement, without any config change. On Thu, Mar 24, 2016 at 1:00 PM Amol Kekre <[email protected]> wrote: > This is a good way, and it fits well with platform self-adjusting. We may > need to still look at a way to allow users to keep SPIN_MILLIS fixed in > case they depend on it for downstream intake. > > Thks > Amol > > > On Thu, Mar 24, 2016 at 12:45 PM, Sandesh Hegde <[email protected]> > wrote: > > > That is cool, we need to something similar for input operators. > > > > On Thu, Mar 24, 2016 at 12:32 PM Thomas Weise <[email protected]> > > wrote: > > > > > Please see: > > > > > > https://issues.apache.org/jira/browse/APEXCORE-380 > > > https://github.com/apache/incubator-apex-core/pull/271 > > > > > > > > > On Thu, Mar 24, 2016 at 12:27 PM, Sandesh Hegde < > [email protected] > > > > > > wrote: > > > > > > > Hello Team, > > > > > > > > The attributes SPIN_MILLIS is the amount of time the input operator > > > sleeps > > > > while there are no inputs to process and it's default value is 10ms. > > > > > > > > Recently while I was doing performance experiments, reducing the > > > > SPIN_MILLIS to 1ms increased the number of tuples processed by the > > input > > > > operator. Instead of manually adjusting the SPIN_MILLIS for best > > > > performance, how about changing it based on the heuristic? > > > > > > > > Here is the idea > > > > After sleeping for SPIN_MILLIS, if tuples are found reduce the > > > SPIN_MILLIS > > > > otherwise increase the SPIN_MILLIS. > > > > > > > > Let me know your thoughts. > > > > > > > > Thanks > > > > Sandesh > > > > > > > > > >
