[
https://issues.apache.org/jira/browse/GEODE-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718310#comment-16718310
]
Dan Smith commented on GEODE-6191:
----------------------------------
Here is the comparison between 48 threads vs 192 threads in google cloud on 4
96 CPU instances. The throughput is similar. One possibility - are we reporting
per thread throughput?
{noformat}
Comparing test result at
/Users/dsmith/Documents/hydrastick_results/./experiment_data/sole2/output-12-04-2018-14-36-22-sole/baseline/benchmarks/
to baseline at
/Users/dsmith/Documents/hydrastick_results/dan_48_threads/output-12-05-2018-16-17-20-dan/baseline/benchmarks
org.apache.geode.benchmark.tests.PartitionedGetBenchmark
average ops/second Baseline: 88476.57 Test: 96893.72
Ratio: 1.10
YS 99th percentile latency Baseline: 1706.00 Test: 5815.00
Ratio: 3.41
median latency Baseline: 158335.00 Test: 177279.00
Ratio: 1.12
90th percentile latency Baseline: 1413119.00 Test: 5357567.00
Ratio: 3.79
99th percentile latency Baseline: 2572287.00 Test: 10485759.00
Ratio: 4.08
99.9th percentile latency Baseline: 13795327.00 Test: 26181631.00
Ratio: 1.90
average latency Baseline: 541350.88 Test: 1979451.14
Ratio: 3.66
latency standard deviation Baseline: 894401.96 Test: 2933674.44
Ratio: 3.28
org.apache.geode.benchmark.tests.PartitionedPutBenchmark
average ops/second Baseline: 80945.34 Test: 80752.05
Ratio: 1.00
YS 99th percentile latency Baseline: 1806.00 Test: 7308.00
Ratio: 4.05
median latency Baseline: 169855.00 Test: 179327.00
Ratio: 1.06
90th percentile latency Baseline: 1523711.00 Test: 6926335.00
Ratio: 4.55
99th percentile latency Baseline: 2795519.00 Test: 13844479.00
Ratio: 4.95
99.9th percentile latency Baseline: 16162815.00 Test: 33079295.00
Ratio: 2.05
average latency Baseline: 591994.63 Test: 2376632.57
Ratio: 4.01
latency standard deviation Baseline: 998543.72 Test: 4008864.70
Ratio: 4.01
org.apache.geode.benchmark.tests.ReplicatedGetBenchmark
average ops/second Baseline: 112914.78 Test: 122716.58
Ratio: 1.09
YS 99th percentile latency Baseline: 1503.00 Test: 5213.00
Ratio: 3.47
median latency Baseline: 150271.00 Test: 173439.00
Ratio: 1.15
90th percentile latency Baseline: 1191935.00 Test: 4771839.00
Ratio: 4.00
99th percentile latency Baseline: 2109439.00 Test: 9314303.00
Ratio: 4.42
99.9th percentile latency Baseline: 14016511.00 Test: 24870911.00
Ratio: 1.77
average latency Baseline: 423987.61 Test: 1563927.52
Ratio: 3.69
latency standard deviation Baseline: 801076.64 Test: 2550712.78
Ratio: 3.18
org.apache.geode.benchmark.tests.ReplicatedPutBenchmark
average ops/second Baseline: 92276.37 Test: 82324.68
Ratio: 0.89
YS 99th percentile latency Baseline: 1202.00 Test: 20029.00
Ratio: 16.66
median latency Baseline: 445439.00 Test: 492287.00
Ratio: 1.11
90th percentile latency Baseline: 684543.00 Test: 7352319.00
Ratio: 10.74
99th percentile latency Baseline: 1204223.00 Test: 11272191.00
Ratio: 9.36
99.9th percentile latency Baseline: 17547263.00 Test: 32440319.00
Ratio: 1.85
average latency Baseline: 519060.95 Test: 2330467.72
Ratio: 4.49
latency standard deviation Baseline: 898028.16 Test: 4439782.10
Ratio: 4.94
{noformat}
> Investigate scaleability of benchmarks for different numbers of threads
> -----------------------------------------------------------------------
>
> Key: GEODE-6191
> URL: https://issues.apache.org/jira/browse/GEODE-6191
> Project: Geode
> Issue Type: Task
> Components: benchmarks
> Reporter: Dan Smith
> Priority: Major
>
> We should expect to see benchmark throughput scale linearly with the number
> of threads, up to the point where we start hitting either CPU or network
> limitations. If we do not scale, that indicates that either something in the
> benchmark framework or Geode itself is limiting us.
> In a couple of runs in google cloud with 48 threads vs 192 threads on 4 96
> CPU instances, we observed almost the same throughput (but which much higher
> latency) with 192 threads. CPU and network stats did not indicate full
> utilization.
> We should check the scaleability of these tests again after GEODE-6172 and
> GEODE-6148 are implemented. Try running the tests with increasing numbers of
> threads (eg 4,16,32,64,128,256,512, etc.) in AWS on c5.9xlarge instances and
> see when we stop scaling linearly and why.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
