[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-22 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749517#comment-16749517
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

Thanks for your feedback [~ascherbakov], i've resolved them.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Fix For: 2.8
>
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-22 Thread Alexei Scherbakov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748633#comment-16748633
 ] 

Alexei Scherbakov commented on IGNITE-10877:


[~voropava],

I left two minor comments in PR. Otherwise looks good.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Fix For: 2.8
>
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-22 Thread Ignite TC Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748615#comment-16748615
 ] 

Ignite TC Bot commented on IGNITE-10877:


{panel:title=-- Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2866643buildTypeId=IgniteTests24Java8_RunAll]

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Fix For: 2.8
>
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-20 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747703#comment-16747703
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

I don't think it breaks compatibilty, cause we have ignite property to rollback 
to original behaviour for mixed envs.

Moveover GridAffinityAssignment serialization is broken right now. See 
IGNITE-10925, we need to fix issue there.

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Fix For: 2.8
>
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-20 Thread Alexei Scherbakov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747443#comment-16747443
 ] 

Alexei Scherbakov commented on IGNITE-10877:


[~voropava],

1. Please always use bounded maps and sets if you know a size in advance. On 
example, in *initPrimaryBackupMaps* you should use U.newHashSet(assignmentSize) 
or even consider using GridLeanMap for maps from 1-5 keys. This should save 
more of heap size.

2. The compatibility seems broken because you have changed 
GridAffinityAssignment and it's transffered between nodes in 
org.apache.ignite.internal.processors.affinity.GridAffinityUtils.AffinityJob. 
If old node will receive new object it will fail on deserialization.

Please create new ticket and address these issues.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Fix For: 2.8
>
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-18 Thread Alexei Scherbakov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746171#comment-16746171
 ] 

Alexei Scherbakov commented on IGNITE-10877:


[~voropava],

My comments:
 # I do not like IGNITE_AFFINITY_BACKUPS_THRESHOLD property. You already know 
number of nodes in assignment and should automatically choose most optimal way 
to store assignements depending on benchmark results.
 # In addition to 1 please introduce new system property like 
IGNITE_DISABLE_AFFINITY_MEMORY_OPTIMIZATION to force old implementation.
 # Please introduce test which automatically compares heap occupation with 
enabled and disabled optimization with various topology/partitions/caches 
sizes. Optimized mode must always be better than previous implementation. You 
can use techniques like ObjectGraphMeasurer for estimating object graph size 
from application.
 # I do not like class BitSetIntSet. It's better to introduce interface 
GridIntSet implementing Set and providing primitive(batch) 
iterators/getters API. In future where might be other implementations, on 
example compressed bit sets (see IGNITE-4554). I think this might be done in 
separate ticket and not crucial for this contribution.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-18 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746083#comment-16746083
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

16part, 16 nodes

HashSet

!image-2019-01-18-12-09-32-876.png!

BitSet

!image-2019-01-18-12-09-04-835.png!

 

In total we have:

N - number of nodes

P - number of parts

low P, low N  - BitSet better

high P, low N - BitSet better

low P, high N - BitSet slightly better

high P, high N - HashSet is better

I suggest to have threshold of 500.

 

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png, 
> image-2019-01-18-12-09-04-835.png, image-2019-01-18-12-09-32-876.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-18 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745931#comment-16745931
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

1024part 4k nodes

HashSet

!image-2019-01-18-11-56-10-339.png!

BitSet

!image-2019-01-18-11-56-18-040.png!

 On as small number of partitions BitSet is roughly the same with HashSet.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png, image-2019-01-18-11-55-39-496.png, 
> image-2019-01-18-11-56-10-339.png, image-2019-01-18-11-56-18-040.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-18 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745911#comment-16745911
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

64k parts, 4k nodes 

HashSets

!image-2019-01-18-11-38-39-410.png!

BitSets

!image-2019-01-18-11-36-57-451.png!

 

We see that HashSet is the clear winner on high number of cluster nodes, 
alongside with high number of partitions.

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png, image-2019-01-18-11-36-57-451.png, 
> image-2019-01-18-11-38-39-410.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-17 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745009#comment-16745009
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

65k partitions 160 nodes

 

!image-2019-01-17-15-45-53-043.png!

!image-2019-01-17-15-46-32-872.png!

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png, 
> image-2019-01-17-15-45-49-561.png, image-2019-01-17-15-45-53-043.png, 
> image-2019-01-17-15-46-32-872.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-17 Thread Ignite TC Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745006#comment-16745006
 ] 

Ignite TC Bot commented on IGNITE-10877:


{panel:title=-- Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2815399buildTypeId=IgniteTests24Java8_RunAll]

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-17 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744853#comment-16744853
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

Here is the result of optimization on :

8K partitions, 200 caches, 20 groups, 4 nodes

Before:

!image-2019-01-17-12-58-07-382.png!

After:

 

!image-2019-01-17-12-59-52-137.png!

We see compaction upto 250times for backup collections.

Moreover if backup less then 6 we won't allocate List> using 
viewReadOnly(ArrayList) to not allocate and keep objects.

 

 

 

 

 

 

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr, 
> image-2019-01-17-12-58-07-382.png, image-2019-01-17-12-59-52-137.png
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-17 Thread Ivan Daschinskiy (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744803#comment-16744803
 ] 

Ivan Daschinskiy commented on IGNITE-10877:
---

Hi, [~voropava], your patch looks good to me, thanks for contribution.

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-16 Thread Ignite TC Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743962#comment-16743962
 ] 

Ignite TC Bot commented on IGNITE-10877:


{panel:title=-- Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2805125buildTypeId=IgniteTests24Java8_RunAll]

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-10877) GridAffinityAssignment.initPrimaryBackupMaps memory pressure

2019-01-15 Thread Pavel Voronkin (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743205#comment-16743205
 ] 

Pavel Voronkin commented on IGNITE-10877:
-

Benchmark Mode Cnt Score Error Units
SmallHashSetsVsReadOnlyViewBenchmark.hashSetContainsRandom thrpt 20 
26221395,193 ± 240929,392 ops/s
SmallHashSetsVsReadOnlyViewBenchmark.hashSetIteratorRandom thrpt 20 
12626598,194 ± 1742223,886 ops/s
SmallHashSetsVsReadOnlyViewBenchmark.readOnlyViewContainsRandom thrpt 20 
23301229,681 ± 534549,170 ops/s
SmallHashSetsVsReadOnlyViewBenchmark.readOnlyViewIteratorRandom thrpt 20 
21134614,093 ± 666488,488 ops/s

we see 2x improvement in iterator and slight reduce in contains() in addition 
to reduced allocations.

 

> GridAffinityAssignment.initPrimaryBackupMaps memory pressure
> 
>
> Key: IGNITE-10877
> URL: https://issues.apache.org/jira/browse/IGNITE-10877
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Voronkin
>Assignee: Pavel Voronkin
>Priority: Major
> Attachments: grid.srv.node.1.0-29.12.2018-12.50.15.jfr
>
>
> 1) While running tests with JFR we observe huge memory allocation pressure 
> produced by:
> *Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)*
> java.util.HashMap.newNode(int, Object, Object, HashMap$Node) 481 298 044 784 
> 100
>  java.util.HashMap.putVal(int, Object, Object, boolean, boolean) 481 298 044 
> 784 100
>  java.util.HashMap.put(Object, Object) 481 298 044 784 100
>  java.util.HashSet.add(Object) 480 297 221 040 99,724
>  
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps()
>  1 823 744 0,276
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.(AffinityTopologyVersion,
>  List, List) 1 823 744 0,276
> *Allocation stats*
> Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB 
> Size(bytes) Total TLAB Size(bytes) Pressure(%)
>  java.util.HashMap$Node 32 15 392 481 619 635,726 298 044 784 32,876
>  java.lang.Object[] 1 470,115 461 616 314 655 019,236 205 676 040 22,687
>  java.util.HashMap$Node[] 41 268,617 6 149 024 149 690 046,067 102 816 864 
> 11,341
>  java.lang.Integer 16 1 456 91 662 911,385 60 324 936 6,654
>  java.util.ArrayList 24 1 608 67 703 389,97 47 127 128 5,198
> 2) Also another hot place found
> Stack Trace TLABs Total TLAB Size(bytes) Pressure(%)
>  java.util.ArrayList.grow(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureExplicitCapacity(int) 7 5 766 448 9,554
>  java.util.ArrayList.ensureCapacityInternal(int) 7 5 766 448 9,554
>  java.util.ArrayList.add(Object) 7 5 766 448 9,554
>  
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl.nodes(int,
>  AffinityTopologyVersion, GridDhtPartitionState, GridDhtPartitionState[]) 7 5 
> 766 448 9,554
> The reason of that is defail
> I think we need to improve memory efficiency by switching from from Sets to 
> BitSets
>  
> JFR attached, see Allocations in 12:50:28 - 12:50:29
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)