Hi Juhyung,
On 2023/4/4 15:36, Juhyung Park wrote:
Hi everyone,
I want to start a discussion on using f2fs for regular desktops/workstations.
There are growing number of interests in using f2fs as the general
root file-system:
2018: https://www.phoronix.com/news/GRUB-Now-Supports-F2FS
2020: https://www.phoronix.com/news/Clear-Linux-F2FS-Root-Option
2023: https://code.launchpad.net/~nexusprism/curtin/+git/curtin/+merge/439880
2023: https://code.launchpad.net/~nexusprism/grub/+git/ubuntu/+merge/440193
I've been personally running f2fs on all of my x86 Linux boxes since
2015, and I have several concerns that I think we need to collectively
address for regular non-Android normies to use f2fs:
A. Bootloader and installer support
B. Host-side GC
C. Extended node bitmap
I'll go through each one.
=== A. Bootloader and installer support ===
It seems that both GRUB and systemd-boot supports f2fs without the
need for a separate ext4-formatted /boot partition.
Some distros are seemingly disabling f2fs module for GRUB though for
security reasons:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868664
It's ultimately up to the distro folks to enable this, and still in
the worst-case scenario, they can specify a separate /boot partition
and format it to ext4 upon installation.
The installer itself to show f2fs and call mkfs.f2fs is being worked
on currently on Ubuntu. See the 2023 links above.
Nothing f2fs mainline developers should do here, imo.
=== B. Host-side GC ===
f2fs relieves most of the device-side GC but introduces a new
host-side GC. This is extremely confusing for people who have no
background in SSDs and flash storage to understand, let alone
discard/trim/erase complications.
In most consumer-grade blackbox SSDs, device-side GCs are handled
automatically for various workloads. f2fs, however, leaves that
responsibility to the userspace with conservative tuning on the
We've proposed a f2fs feature named "space awared garbage collection"
and shipped it in huawei/honor's devices, but forgot to try upstreaming
it. :-P
In this feature, we introduced three mode:
- performance mode: something like write-gc in ftl, it can trigger
background gc more frequently and tune its speed according to free
segs and reclaimable blks ratio.
- lifetime mode: slow down background gc to avoid high waf if there
is less free space.
- balance mode: behave as usual.
I guess this may be helpful for Linux desktop distros since there is
no such storage service trigger gc_urgent.
kernel-side by default. Android handles this by init.rc tunings and a
separate code running in vold to trigger gc_urgent.
For regular Linux desktop distros, f2fs just runs on the default
configuration set on the kernel and unless it’s running 24/7 with
plentiful idle time, it quickly runs out of free segments and starts
triggering foreground GC. This is giving people the wrong impression
that f2fs slows down far drastically than other file-systems when
that’s quite the contrary (i.e., less fragmentation overtime).
This is almost the equivalent of re-living the nightmare of trim. On
SSDs with very small to no over-provisioned space, running a
file-system with no discard what-so-ever (sadly still a common case
when an external SSD is used with no UAS) will also drastically slow
What does UAS mean?
the performance down. On file-systems with no asynchronous discard,
There is no such performance issue in f2fs, right? as f2fs enables
discard mount option by default, and supports async discard feature.
mounting a file-system with the discard option adds a non-negligible
overhead on every remove/delete operations, so most distros now
(thankfully) use a timer job registered to systemd to trigger fstrim:
https://github.com/util-linux/util-linux/commits/master/sys-utils/fstrim.timer
This is still far from ideal. The default file-system, ext4, slows
down drastically almost to a halt when fstrim -a is called, especially
on SATA. For some reason that is still a mystery for me, people seem
to be happy with it. No one bothered to improve it for years
¯\_(ツ)_/¯.
So here’s my proposal:
As Linux distros don’t have a good mechanism for hinting when to
trigger GC, introduce a new Kconfig, CONFIG_F2FS_GC_UPON_FSTRIM and
enable it by default.
This config will hook up ioctl(FITRIM), which is currently ignored on
f2fs -
https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=master&id=e555da9f31210d2b62805cd7faf29228af7c3cfb
, to perform discard and GC on all invalid segments.
Userspace configuration with enough f2fs/GC knowledge such as Android
should disable it.
This will ensure that Linux distros that blindly call fstrim will at
least avoid constant slowdowns when free segments are depleted with
the occasional (once a week) slowdown, which *people are already
living with on ext4*. I'll even go further and mention that since f2fs
GC is a regular R/W workload, it doesn't cause an extreme slowdown
comparable to a level of a full file-system trim operation.
If this is acceptable, I’ll cook up a patch.
In an ideal world, all Linux distros should have an explicit f2fs GC
trigger mechanism (akin to
https://github.com/kdave/btrfsmaintenance#distro-integration ), but
it’s practically unrealistic to expect that, given the installer
doesn’t even support f2fs for now.
=== C. Extended node bitmap ===
f2fs by default have a very limited number of allowed inodes compared
to other file-systems. Just 2 AOSP syncs are enough to exhaust f2fs
and result in -ENOSPC.
Here are some of the stats collected from me and my colleague that we
use daily as a regular desktop with GUI, web-browsing and everything:
1. Laptop
Utilization: 68% (182914850 valid blocks, 462 discard blocks)
- Node: 10234905 (Inode: 10106526, Other: 128379)
- Data: 172679945
- Inline_xattr Inode: 2004827
- Inline_data Inode: 867204
- Inline_dentry Inode: 51456
2. Desktop #1
Utilization: 55% (133310465 valid blocks, 0 discard blocks)
- Node: 6389660 (Inode: 6289765, Other: 99895)
- Data: 126920805
- Inline_xattr Inode: 2253838
- Inline_data Inode: 1119109
- Inline_dentry Inode: 187958
3. Desktop #2
Utilization: 83% (202222003 valid blocks, 1 discard blocks)
- Node: 21887836 (Inode: 21757139, Other: 130697)
- Data: 180334167
- Inline_xattr Inode: 39292
- Inline_data Inode: 35213
- Inline_dentry Inode: 1127
4. Colleague
Utilization: 22% (108652929 valid blocks, 362420605 discard blocks)
- Node: 5629348 (Inode: 5542909, Other: 86439)
- Data: 103023581
- Inline_xattr Inode: 655752
- Inline_data Inode: 259900
- Inline_dentry Inode: 193000
5. Android phone (for reference)
Utilization: 78% (36505713 valid blocks, 1074 discard blocks)
- Node: 704698 (Inode: 683337, Other: 21361)
- Data: 35801015
- Inline_xattr Inode: 683333
- Inline_data Inode: 237470
- Inline_dentry Inode: 112177
Chao Yu added a functionality to expand this via the -i flag passed to
mkfs.f2fs back in 2018 -
https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/commit/?id=baaa076b4d576042913cfe34169442dfda651ca4
I occasionally find myself in a weird position of having to tell
people "Oh you should use the -i option from mkfs.f2fs" when they
encounter this issue only after they’ve migrated most of the data and
ask back "Why isn’t this enabled by default?".
While this might not be an issue for the foreseeable future in
Android, I’d argue that this is a feature that needs to be enabled by
default for desktop environments with preferably a robust testing
Yes, I guess we need to add some testcases and do some robust tests for
large nat_bitmap feature first. Since I do remember once its design flaw
corrupted data. :(
Thanks,
infrastructure. Guarding this with #ifndef __ANDROID__ doesn’t seem to
make much sense as it introduces more complications to how
fuzzing/testing should be done.
I’ll also add that it’s a common practice for userspace mkfs tools to
introduce breaking default changes to older kernels (with options to
produce a legacy image, of course).
This was a lengthy email, but I hope I was being reasonable.
Jaegeuk and Chao, let me know what you think.
And as always, thanks for your hard work :)
Thanks,
regards
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
