[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074124#comment-14074124
]
graham sanderson commented on CASSANDRA-7546:
---------------------------------------------
Once again numbers - note I'm still using the same test driver as before (hence
the 0 up/down, count numbers etc), though I have updated it to pass a column
cloner in the transform.
{code}
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 64
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 1020ms maxConcurrency = 1
[junit] GC for PS Scavenge: 37 ms for 3 collections
[junit] Approx allocation = 589MB vs 8MB; ratio to raw data size =
73.61468285714285
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 963ms maxConcurrency = 1
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 584MB vs 8MB; ratio to raw data size =
72.99738571428571
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 826ms maxConcurrency = 1
[junit] GC for PS Scavenge: 24 ms for 2 collections
[junit] Approx allocation = 496MB vs 8MB; ratio to raw data size =
61.99165047619048
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 746ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 477MB vs 8MB; ratio to raw data size =
59.63136380952381
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 617ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 362MB vs 8MB; ratio to raw data size =
45.24315523809524
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 602ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 366MB vs 8MB; ratio to raw data size =
45.77833523809524
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 443ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 308MB vs 8MB; ratio to raw data size =
38.468846666666664
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 422ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 309MB vs 8MB; ratio to raw data size =
38.667831428571425
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 64
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 2039ms maxConcurrency = 100
[junit] GC for PS Scavenge: 118 ms for 34 collections
[junit] Approx allocation = 11178MB vs 8MB; ratio to raw data size =
1395.417500952381
[junit] loopRatio (closest to 1 best) 18.20478 raw 100000/1820478 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 1299ms maxConcurrency = 100
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 614MB vs 8MB; ratio to raw data size =
76.68355047619048
[junit] loopRatio (closest to 1 best) 1.05291 raw 779/6045 counted 0/0 sync
99246/99246 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 224ms maxConcurrency = 100
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 832MB vs 8MB; ratio to raw data size =
103.97120666666666
[junit] loopRatio (closest to 1 best) 1.89634 raw 100000/189634 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 226ms maxConcurrency = 99
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 810MB vs 8MB; ratio to raw data size =
101.20042857142857
[junit] loopRatio (closest to 1 best) 1.92036 raw 99094/191116 counted 0/0
sync 920/920 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 188ms maxConcurrency = 99
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 368MB vs 8MB; ratio to raw data size =
46.03569619047619
[junit] loopRatio (closest to 1 best) 1.02239 raw 100000/102239 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 185ms maxConcurrency = 98
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 376MB vs 8MB; ratio to raw data size =
46.975274285714285
[junit] loopRatio (closest to 1 best) 1.02077 raw 100000/102077 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 183ms maxConcurrency = 98
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 309MB vs 8MB; ratio to raw data size =
38.642466666666664
[junit] loopRatio (closest to 1 best) 1.00435 raw 100000/100435 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 189ms maxConcurrency = 97
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 315MB vs 8MB; ratio to raw data size =
39.396990476190474
[junit] loopRatio (closest to 1 best) 1.00374 raw 100000/100374 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 256
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 773ms maxConcurrency = 1
[junit] GC for PS Scavenge: 20 ms for 2 collections
[junit] Approx allocation = 618MB vs 26MB; ratio to raw data size =
23.51672376811594
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 375ms maxConcurrency = 1
[junit] GC for PS Scavenge: 19 ms for 2 collections
[junit] Approx allocation = 618MB vs 26MB; ratio to raw data size =
23.48319043478261
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 330ms maxConcurrency = 1
[junit] GC for PS Scavenge: 8 ms for 1 collections
[junit] Approx allocation = 506MB vs 26MB; ratio to raw data size =
19.239044057971014
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 348ms maxConcurrency = 1
[junit] GC for PS Scavenge: 8 ms for 1 collections
[junit] Approx allocation = 507MB vs 26MB; ratio to raw data size =
19.28066347826087
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 709ms maxConcurrency = 1
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 393MB vs 26MB; ratio to raw data size =
14.948667826086956
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 619ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 407MB vs 26MB; ratio to raw data size =
15.49846347826087
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 600ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 343MB vs 26MB; ratio to raw data size =
13.053241739130435
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 551ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 346MB vs 26MB; ratio to raw data size =
13.151993623188405
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 256
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 2377ms maxConcurrency = 100
[junit] GC for PS Scavenge: 131 ms for 35 collections
[junit] Approx allocation = 11554MB vs 26MB; ratio to raw data size =
438.97863014492754
[junit] loopRatio (closest to 1 best) 18.38368 raw 100000/1838368 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 1399ms maxConcurrency = 100
[junit] GC for PS Scavenge: 31 ms for 2 collections
[junit] Approx allocation = 634MB vs 26MB; ratio to raw data size =
24.100446666666667
[junit] loopRatio (closest to 1 best) 1.05401 raw 1016/6325 counted 0/0 sync
99075/99076 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 237ms maxConcurrency = 99
[junit] GC for PS Scavenge: 33 ms for 3 collections
[junit] Approx allocation = 880MB vs 26MB; ratio to raw data size =
33.46811043478261
[junit] loopRatio (closest to 1 best) 1.90156 raw 100000/190156 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 220ms maxConcurrency = 100
[junit] GC for PS Scavenge: 21 ms for 2 collections
[junit] Approx allocation = 834MB vs 26MB; ratio to raw data size =
31.701786376811594
[junit] loopRatio (closest to 1 best) 1.89694 raw 99747/189437 counted 0/0
sync 256/257 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 187ms maxConcurrency = 100
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 401MB vs 26MB; ratio to raw data size =
15.241803188405797
[junit] loopRatio (closest to 1 best) 1.02141 raw 100000/102141 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 183ms maxConcurrency = 100
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 408MB vs 26MB; ratio to raw data size =
15.502840869565217
[junit] loopRatio (closest to 1 best) 1.02057 raw 100000/102057 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 179ms maxConcurrency = 98
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 346MB vs 26MB; ratio to raw data size =
13.181417391304349
[junit] loopRatio (closest to 1 best) 1.00479 raw 100000/100479 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 189ms maxConcurrency = 97
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 350MB vs 26MB; ratio to raw data size =
13.331782608695653
[junit] loopRatio (closest to 1 best) 1.00373 raw 100000/100373 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 1024
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 1024ms maxConcurrency = 1
[junit] GC for PS Scavenge: 35 ms for 3 collections
[junit] Approx allocation = 722MB vs 99MB; ratio to raw data size =
7.260372490421456
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 799ms maxConcurrency = 1
[junit] GC for PS Scavenge: 34 ms for 3 collections
[junit] Approx allocation = 757MB vs 99MB; ratio to raw data size =
7.612250574712644
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 562ms maxConcurrency = 1
[junit] GC for PS Scavenge: 32 ms for 3 collections
[junit] Approx allocation = 641MB vs 99MB; ratio to raw data size =
6.442458850574712
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 528ms maxConcurrency = 1
[junit] GC for PS Scavenge: 32 ms for 3 collections
[junit] Approx allocation = 651MB vs 99MB; ratio to raw data size =
6.5402478927203065
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 665ms maxConcurrency = 1
[junit] GC for PS Scavenge: 26 ms for 2 collections
[junit] Approx allocation = 559MB vs 99MB; ratio to raw data size =
5.618937088122605
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 815ms maxConcurrency = 1
[junit] GC for PS Scavenge: 28 ms for 2 collections
[junit] Approx allocation = 547MB vs 99MB; ratio to raw data size =
5.494165977011495
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 686ms maxConcurrency = 1
[junit] GC for PS Scavenge: 28 ms for 2 collections
[junit] Approx allocation = 490MB vs 99MB; ratio to raw data size =
4.927921072796935
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 702ms maxConcurrency = 1
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 483MB vs 99MB; ratio to raw data size =
4.851513026819924
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0 sync
0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 1024
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 2462ms maxConcurrency = 100
[junit] GC for PS Scavenge: 160 ms for 45 collections
[junit] Approx allocation = 14795MB vs 99MB; ratio to raw data size =
148.60839471264367
[junit] loopRatio (closest to 1 best) 18.92911 raw 100000/1892911 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 1333ms maxConcurrency = 100
[junit] GC for PS Scavenge: 26 ms for 2 collections
[junit] Approx allocation = 763MB vs 99MB; ratio to raw data size =
7.6639096551724135
[junit] loopRatio (closest to 1 best) 1.05239 raw 861/6055 counted 0/0 sync
99184/99184 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 249ms maxConcurrency = 100
[junit] GC for PS Scavenge: 38 ms for 4 collections
[junit] Approx allocation = 1158MB vs 99MB; ratio to raw data size =
11.633376168582375
[junit] loopRatio (closest to 1 best) 1.98465 raw 100000/198465 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 244ms maxConcurrency = 100
[junit] GC for PS Scavenge: 44 ms for 4 collections
[junit] Approx allocation = 1004MB vs 99MB; ratio to raw data size =
10.08440398467433
[junit] loopRatio (closest to 1 best) 1.92636 raw 99467/192098 counted 0/0
sync 537/538 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 207ms maxConcurrency = 97
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 504MB vs 99MB; ratio to raw data size =
5.063335938697318
[junit] loopRatio (closest to 1 best) 1.02225 raw 100000/102225 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 198ms maxConcurrency = 98
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 558MB vs 99MB; ratio to raw data size =
5.614166206896551
[junit] loopRatio (closest to 1 best) 1.02187 raw 100000/102187 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 214ms maxConcurrency = 98
[junit] GC for PS Scavenge: 27 ms for 2 collections
[junit] Approx allocation = 481MB vs 99MB; ratio to raw data size =
4.832893256704981
[junit] loopRatio (closest to 1 best) 1.00487 raw 100000/100487 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 202ms maxConcurrency = 97
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 500MB vs 99MB; ratio to raw data size =
5.025245363984674
[junit] loopRatio (closest to 1 best) 1.00395 raw 100000/100395 counted 0/0
sync 0/0 up 0 down 0
{code}
> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: graham sanderson
> Assignee: graham sanderson
> Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt,
> 7546.20_4.txt, 7546.20_alt.txt, suggestion1.txt, suggestion1_21.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
