[jira] [Commented] (IGNITE-5884) Change default pageSize of page memory to 4KB

[Sergey Chugunov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158298#comment-16158298
 ] 

Sergey Chugunov commented on IGNITE-5884:
-

[~ivan.glukos], the change looks good to me, TC status is acceptable (don't see 
any test failures related to the change). Please proceed with merging.

> Change default pageSize of page memory to 4KB
> -
>
> Key: IGNITE-5884
> URL: https://issues.apache.org/jira/browse/IGNITE-5884
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Critical
>  Labels: usability
> Fix For: 2.3
>
> Attachments: CpBenchmark.java, iostat.log, ssdlab.log
>
>
> Checkpoint write speed is suboptimal with default 2K page on most UNIX-driven 
> enviroments with SSD disk. There are several reasons for this:
> 1) Page size of linux page cache is 4k by default on most kernels (you can 
> check yours by "getconf PAGE_SIZE" command). With 2k random writes 
> vm.dirty_ratio threshold is reached two times faster than with 4k random 
> writes.
> 2) Most SSD manufacturers don't expose actual disk page size, but they 
> recommend to write at least 4k at once. Also, 4k blocks are used during 
> benchmarking SSD random writes. 
> Related question: 
> https://superuser.com/questions/1168014/nvme-ssd-why-is-4k-writing-faster-than-reading
> Article by Emmanuel Goossaert describing why writing less than a page is 
> сounterproductive: 
> http://codecapsule.com/2014/02/12/coding-for-ssds-part-3-pages-blocks-and-the-flash-translation-layer/
> I've prepared a checkpoint emulation benchmark (code and results attached). 
> Run on production-level hardware (CentOS, 100 GB RAM, total LFS size is 
> 100GB, vm.dirty_ratio=10) showed that checkpointing with 4k pages is much 
> more efficient than with 2k.
> *Important: backwards compatibility must be ensured with LFS files created 
> with old 2k default page size.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5884) Change default pageSize of page memory to 4KB

[Ivan Rakov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148984#comment-16148984
 ] 

Ivan Rakov commented on IGNITE-5884:


TC run: https://ci.ignite.apache.org/viewQueued.html?itemId=803028

> Change default pageSize of page memory to 4KB
> -
>
> Key: IGNITE-5884
> URL: https://issues.apache.org/jira/browse/IGNITE-5884
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Critical
>  Labels: usability
> Fix For: 2.3
>
> Attachments: CpBenchmark.java, iostat.log, ssdlab.log
>
>
> Checkpoint write speed is suboptimal with default 2K page on most UNIX-driven 
> enviroments with SSD disk. There are several reasons for this:
> 1) Page size of linux page cache is 4k by default on most kernels (you can 
> check yours by "getconf PAGE_SIZE" command). With 2k random writes 
> vm.dirty_ratio threshold is reached two times faster than with 4k random 
> writes.
> 2) Most SSD manufacturers don't expose actual disk page size, but they 
> recommend to write at least 4k at once. Also, 4k blocks are used during 
> benchmarking SSD random writes. 
> Related question: 
> https://superuser.com/questions/1168014/nvme-ssd-why-is-4k-writing-faster-than-reading
> Article by Emmanuel Goossaert describing why writing less than a page is 
> сounterproductive: 
> http://codecapsule.com/2014/02/12/coding-for-ssds-part-3-pages-blocks-and-the-flash-translation-layer/
> I've prepared a checkpoint emulation benchmark (code and results attached). 
> Run on production-level hardware (CentOS, 100 GB RAM, total LFS size is 
> 100GB, vm.dirty_ratio=10) showed that checkpointing with 4k pages is much 
> more efficient than with 2k.
> *Important: backwards compatibility must be ensured with LFS files created 
> with old 2k default page size.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5884) Change default pageSize of page memory to 4KB

[Ivan Rakov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148976#comment-16148976
 ] 

Ivan Rakov commented on IGNITE-5884:


Please, review.

> Change default pageSize of page memory to 4KB
> -
>
> Key: IGNITE-5884
> URL: https://issues.apache.org/jira/browse/IGNITE-5884
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Critical
>  Labels: usability
> Fix For: 2.3
>
> Attachments: CpBenchmark.java, iostat.log, ssdlab.log
>
>
> Checkpoint write speed is suboptimal with default 2K page on most UNIX-driven 
> enviroments with SSD disk. There are several reasons for this:
> 1) Page size of linux page cache is 4k by default on most kernels (you can 
> check yours by "getconf PAGE_SIZE" command). With 2k random writes 
> vm.dirty_ratio threshold is reached two times faster than with 4k random 
> writes.
> 2) Most SSD manufacturers don't expose actual disk page size, but they 
> recommend to write at least 4k at once. Also, 4k blocks are used during 
> benchmarking SSD random writes. 
> Related question: 
> https://superuser.com/questions/1168014/nvme-ssd-why-is-4k-writing-faster-than-reading
> Article by Emmanuel Goossaert describing why writing less than a page is 
> сounterproductive: 
> http://codecapsule.com/2014/02/12/coding-for-ssds-part-3-pages-blocks-and-the-flash-translation-layer/
> I've prepared a checkpoint emulation benchmark (code and results attached). 
> Run on production-level hardware (CentOS, 100 GB RAM, total LFS size is 
> 100GB, vm.dirty_ratio=10) showed that checkpointing with 4k pages is much 
> more efficient than with 2k.
> *Important: backwards compatibility must be ensured with LFS files created 
> with old 2k default page size.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5884) Change default pageSize of page memory to 4KB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148975#comment-16148975
 ] 

ASF GitHub Bot commented on IGNITE-5884:


GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/2565

IGNITE-5884 Change default pageSize of page memory to 4KB



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5884

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2565.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2565


commit a4bcc6f3946462b29c2d1dea325a8ee037772a4c
Author: Ivan Rakov 
Date:   2017-08-31T13:30:34Z

IGNITE-5884 Change default pageSize of page memory to 4KB




> Change default pageSize of page memory to 4KB
> -
>
> Key: IGNITE-5884
> URL: https://issues.apache.org/jira/browse/IGNITE-5884
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Critical
>  Labels: usability
> Fix For: 2.3
>
> Attachments: CpBenchmark.java, iostat.log, ssdlab.log
>
>
> Checkpoint write speed is suboptimal with default 2K page on most UNIX-driven 
> enviroments with SSD disk. There are several reasons for this:
> 1) Page size of linux page cache is 4k by default on most kernels (you can 
> check yours by "getconf PAGE_SIZE" command). With 2k random writes 
> vm.dirty_ratio threshold is reached two times faster than with 4k random 
> writes.
> 2) Most SSD manufacturers don't expose actual disk page size, but they 
> recommend to write at least 4k at once. Also, 4k blocks are used during 
> benchmarking SSD random writes. 
> Related question: 
> https://superuser.com/questions/1168014/nvme-ssd-why-is-4k-writing-faster-than-reading
> Article by Emmanuel Goossaert describing why writing less than a page is 
> сounterproductive: 
> http://codecapsule.com/2014/02/12/coding-for-ssds-part-3-pages-blocks-and-the-flash-translation-layer/
> I've prepared a checkpoint emulation benchmark (code and results attached). 
> Run on production-level hardware (CentOS, 100 GB RAM, total LFS size is 
> 100GB, vm.dirty_ratio=10) showed that checkpointing with 4k pages is much 
> more efficient than with 2k.
> *Important: backwards compatibility must be ensured with LFS files created 
> with old 2k default page size.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5884) Change default pageSize of page memory to 4KB

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133558#comment-16133558
 ] 

Denis Magda commented on IGNITE-5884:
-

Discussion on the @dev list: 
http://apache-ignite-developers.2346864.n4.nabble.com/Default-page-size-must-be-changed-to-4k-Should-it-be-backwards-compatible-td20865.html#a20874

> Change default pageSize of page memory to 4KB
> -
>
> Key: IGNITE-5884
> URL: https://issues.apache.org/jira/browse/IGNITE-5884
> Project: Ignite
>  Issue Type: Improvement
>  Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Critical
>  Labels: usability
> Fix For: 2.2
>
> Attachments: CpBenchmark.java, iostat.log, ssdlab.log
>
>
> Checkpoint write speed is suboptimal with default 2K page on most UNIX-driven 
> enviroments with SSD disk. There are several reasons for this:
> 1) Page size of linux page cache is 4k by default on most kernels (you can 
> check yours by "getconf PAGE_SIZE" command). With 2k random writes 
> vm.dirty_ratio threshold is reached two times faster than with 4k random 
> writes.
> 2) Most SSD manufacturers don't reveal actual disk page size, but they 
> recommend to write at least 4k at once. Also, 4k blocks are used during 
> benchmarking SSD random writes. 
> Related question: 
> https://superuser.com/questions/1168014/nvme-ssd-why-is-4k-writing-faster-than-reading
> Article by Emmanuel Goossaert describing why writing less than a page is 
> сounterproductive: 
> http://codecapsule.com/2014/02/12/coding-for-ssds-part-3-pages-blocks-and-the-flash-translation-layer/
> I've prepared a checkpoint emulation benchmark (code and results attached). 
> Run on production-level hardware (CentOS, 100 GB RAM, total LFS size is 
> 100GB, vm.dirty_ratio=10) showed that checkpointing with 4k pages is much 
> more efficient than with 2k.
> *Important: backwards compatibility must be ensured with LFS files created 
> with old 2k default page size.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)