[ https://issues.apache.org/jira/browse/IGNITE-6532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aleksey Plekhanov updated IGNITE-6532: -------------------------------------- Fix Version/s: (was: 2.9) 2.10 > Introduce preallocation in LFS files to avoid high fragmentation on > filesystem level > ------------------------------------------------------------------------------------ > > Key: IGNITE-6532 > URL: https://issues.apache.org/jira/browse/IGNITE-6532 > Project: Ignite > Issue Type: Bug > Components: persistence > Affects Versions: 2.2 > Reporter: Ivan Rakov > Priority: Major > Fix For: 2.10 > > > Modern databases (Oracle, MySql) work with storage drive on physical level, > creating their own partition table and filesystem. > Ignite Persistent Store work with regular files. It appends new pages to > partition file once new pages are allocated and written on checkpoint. These > new pages can form one or several fragments on filesystem level. > As a result, after weeks of uptime, partition files can contain huge amount > of fragments. There were reports about 1200000 fragments in index.bin file on > XFS filesystem. > We can work this around by preallocating files in bigger chunks, e.g. 1000 > pages at a time. On the other hand, early allocation will increase LFS size > overhead, so we should consider reasonable heuristic for allocation. > Allocation should be performed on native level. Just writing a byte at > position (file_size + page_size * 1000) won't do it because XFS (and other > filesystems as well) has an optimization for that case. Missing range will be > just skipped. > Related article about filesystem internals: > https://blog.codecentric.de/en/2017/04/xfs-possible-memory-allocation-deadlock-kmem_alloc/ -- This message was sent by Atlassian Jira (v8.3.4#803005)