[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[
https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863018#comment-16863018
]
Ignite TC Bot commented on IGNITE-10913:
{panel:title=--> Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Core Linux){color} [[tests 2 TIMEOUT
|https://ci.ignite.apache.org/viewLog.html?buildId=4086454]]
{color:#d04437}Platform .NET{color} [[tests 0 TIMEOUT
|https://ci.ignite.apache.org/viewLog.html?buildId=4086453]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4086482&buildTypeId=IgniteTests24Java8_RunAll]
> Reduce heap occupation by
> o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
>
>
> Key: IGNITE-10913
> URL: https://issues.apache.org/jira/browse/IGNITE-10913
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Denis Chudov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> With large topology and large amount of caches/partitions and enabled
> persistence could be millions of FilePageStore objects in heap (for each
> partition).
> Each instance has a reference to a File (field cfgFile) storing as String
> absolute path to a partition.
> Also internal File inplementation (on example UnixFile) also allocates space
> for file path.
> I observed about 2Gb of heap space occupied by these objects in one of
> environments.
> Solution: dereference (set to null) cfgFile after object creation, create
> File object lazily on demand when needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[
https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857628#comment-16857628
]
Ignite TC Bot commented on IGNITE-10913:
{panel:title=--> Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric
|https://ci.ignite.apache.org/viewLog.html?buildId=4048798]]
{color:#d04437}Platform .NET (Core Linux){color} [[tests 1 TIMEOUT
|https://ci.ignite.apache.org/viewLog.html?buildId=4048797]]
{color:#d04437}PDS (Compatibility){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4048788]]
{color:#d04437}Platform .NET{color} [[tests 0 TIMEOUT
|https://ci.ignite.apache.org/viewLog.html?buildId=4048796]]
{color:#d04437}Queries 2{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4048736]]
{color:#d04437}[Check Code Style]{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4048824]]
{color:#d04437}Basic 3{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4048754]]
{color:#d04437}PDS 1{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4048792]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4048825&buildTypeId=IgniteTests24Java8_RunAll]
> Reduce heap occupation by
> o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
>
>
> Key: IGNITE-10913
> URL: https://issues.apache.org/jira/browse/IGNITE-10913
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Denis Chudov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> With large topology and large amount of caches/partitions and enabled
> persistence could be millions of FilePageStore objects in heap (for each
> partition).
> Each instance has a reference to a File (field cfgFile) storing as String
> absolute path to a partition.
> Also internal File inplementation (on example UnixFile) also allocates space
> for file path.
> I observed about 2Gb of heap space occupied by these objects in one of
> environments.
> Solution: dereference (set to null) cfgFile after object creation, create
> File object lazily on demand when needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856725#comment-16856725 ] Denis Chudov commented on IGNITE-10913: --- Performance check detected no significant difference with master. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852884#comment-16852884 ] Eduard Shangareev commented on IGNITE-10913: Looks good. Thank you for contribution. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845042#comment-16845042 ] Eduard Shangareev commented on IGNITE-10913: I have left some comments in PR. [~Denis Chudov], please, take a look. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844078#comment-16844078 ] Denis Chudov commented on IGNITE-10913: --- This PR should not be merged to master before passing performance tests. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833809#comment-16833809 ] Denis Chudov commented on IGNITE-10913: --- [~antkr], your notice is correct. I removed this property. [~agoncharuk], please review > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16829308#comment-16829308 ] Anton Kurbanov commented on IGNITE-10913: - [~Denis Chudov], LGTM, except one thing, do we really need new property IGNITE_ENABLE_FILE_STORE_HEAP_OPTIMIZATION? Now it looks like it should be enabled by default. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826438#comment-16826438 ] Denis Chudov commented on IGNITE-10913: --- [~antkr], please review > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[ https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826436#comment-16826436 ] Denis Chudov commented on IGNITE-10913: --- Blocker is not related to this commit. > Reduce heap occupation by > o.a.i.i.processors.cache.persistence.file.FilePageStore instances. > > > Key: IGNITE-10913 > URL: https://issues.apache.org/jira/browse/IGNITE-10913 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > With large topology and large amount of caches/partitions and enabled > persistence could be millions of FilePageStore objects in heap (for each > partition). > Each instance has a reference to a File (field cfgFile) storing as String > absolute path to a partition. > Also internal File inplementation (on example UnixFile) also allocates space > for file path. > I observed about 2Gb of heap space occupied by these objects in one of > environments. > Solution: dereference (set to null) cfgFile after object creation, create > File object lazily on demand when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10913) Reduce heap occupation by o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
[
https://issues.apache.org/jira/browse/IGNITE-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826433#comment-16826433
]
Ignite TC Bot commented on IGNITE-10913:
{panel:title=--> Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}[Javadoc]{color} [[tests 0 BuildFailureOnMessage
|https://ci.ignite.apache.org/viewLog.html?buildId=3700048]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=3690255&buildTypeId=IgniteTests24Java8_RunAll]
> Reduce heap occupation by
> o.a.i.i.processors.cache.persistence.file.FilePageStore instances.
>
>
> Key: IGNITE-10913
> URL: https://issues.apache.org/jira/browse/IGNITE-10913
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Denis Chudov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With large topology and large amount of caches/partitions and enabled
> persistence could be millions of FilePageStore objects in heap (for each
> partition).
> Each instance has a reference to a File (field cfgFile) storing as String
> absolute path to a partition.
> Also internal File inplementation (on example UnixFile) also allocates space
> for file path.
> I observed about 2Gb of heap space occupied by these objects in one of
> environments.
> Solution: dereference (set to null) cfgFile after object creation, create
> File object lazily on demand when needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
