Github user vrozov commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/173#issuecomment-160690609
The larger queue size for Spsc and CircularBuffer Reservoirs is driven by
the 10 milliseconds wait time as waits for not empty and not full conditions
become more frequent with the smaller queue sizes. CPU utilization for smaller
queue sizes does not mean lower cost of producing or consuming events, it is
more frequent and overlapping sleeps both in the producer and in the consumer
threads.
Currently, 10 milliseconds sleep time is the default sleep time for the
Reservoir consumer and can be configured using SPIN_MILLIS attribute. Sleep on
the producer side is hardcoded to 10 milliseconds. After I recompiled
Reservoirs with 0 sleep time in the producer and set SPIN_MILLIS to 0, Spsc
averages 13 million events/s and Circular buffer averages 5 million events/s
and both have >98% CPU utilization. This benchmark suggests that it is
necessary to provide ability for an application to specify sleep time not only
for the consumer but also for the producer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
