[
https://issues.apache.org/jira/browse/SAMZA-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated SAMZA-245:
----------------------------------
Attachment:
org.apache.samza.test.performance.TestSamzaContainerPerformance.SAMZA-245-3.html
org.apache.samza.test.performance.TestSamzaContainerPerformance.master.html
Attaching initial performance test results.
I ran:
{noformat}
$ ./gradlew :samza-test:test -Dsamza.test.single=TestSamzaContainerPerformance
{noformat}
On my MacBook Air:
{noformat}
Model Name: MacBook Air
Model Identifier: MacBookAir5,2
Processor Name: Intel Core i7
Processor Speed: 2 GHz
Number of Processors: 1
Total Number of Cores: 2
L2 Cache (per Core): 256 KB
L3 Cache: 4 MB
Memory: 8 GB
{noformat}
With SAMZA-245-3 patch applied: after the initial warm-up (~5s), the test
processes 9830000 messages in 13s (18s-5s). This comes out to ~756,000
messages/sec (9860000/12).
With master: there doesn't appear to be much warm-up. The test processes
10000000 messages in 37s. This comes out to ~270,000 messages/sec
(10000000/37). If you discount the first 5 seconds to make it fair, then the
test processes ~273,000 messages/sec (8740000/32).
Given the 2.8x difference in performance, I'm a bit concerned that I've broken
something. All of the existing tests pass, but I'm going to write some
SystemConsumers unit tests to validate things.
> Improve SystemConsumers performance
> -----------------------------------
>
> Key: SAMZA-245
> URL: https://issues.apache.org/jira/browse/SAMZA-245
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
> Fix For: 0.8.0
>
> Attachments: SAMZA-245-1.patch, SAMZA-245-3.patch, SAMZA-245.0.patch,
> org.apache.samza.test.performance.TestSamzaContainerPerformance.SAMZA-245-3.html,
> org.apache.samza.test.performance.TestSamzaContainerPerformance.master.html
>
>
> As part of SAMZA-220, a more radical patch was proposed. This patch appears
> to improve SystemConsumers' performance pretty significantly, while also
> reducing its complexity. The decision was made to move this change into the
> 0.8.0 release, rather than the 0.7.0 release, since it's a fairly risky
> change.
> This ticket is to explore updating SystemConsumers to eliminate almost all
> loops in order to increase performance in the Samza container.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
