[PATCH v4 rebased 1/2] fs/erofs: Add support for EROFS

2024-02-19 Thread Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.

This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.

EROFS compressed files will be supported later since it has more work to
polish.

[1] https://erofs.docs.kernel.org

Reviewed-by: Glenn Washburn 
Reviewed-by: Gao Xiang 
Signed-off-by: Yifan Zhao 
---
Resend a rebased version of v4 as a thread and friendly ping... 

 INSTALL |   8 +-
 Makefile.util.def   |   1 +
 docs/grub.texi  |   3 +-
 grub-core/Makefile.core.def |   5 +
 grub-core/fs/erofs.c| 968 
 grub-core/kern/misc.c   |  14 +
 include/grub/misc.h |   1 +
 7 files changed, 995 insertions(+), 5 deletions(-)
 create mode 100644 grub-core/fs/erofs.c

diff --git a/INSTALL b/INSTALL
index 8d9207c84..545015ba2 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
 
 * If running a Linux kernel the following modules must be loaded:
   - fuse, loop
-  - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+  - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
 reiserfs, udf, xfs
   - On newer kernels, the exfat kernel modules may be used instead of the
 exfat FUSE filesystem
 * The following are Debian named packages required mostly for the full
   suite of filesystem testing (but some are needed by other tests as well):
-  - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+  - btrfs-progs, dosfstools, erofs-utils, e2fsprogs, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
   - exfat-fuse, if not using the exfat kernel module
   - gzip, lzop, xz-utils
   - attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
   common = grub-core/fs/cpio_be.c;
   common = grub-core/fs/odc.c;
   common = grub-core/fs/newc.c;
+  common = grub-core/fs/erofs.c;
   common = grub-core/fs/ext2.c;
   common = grub-core/fs/fat.c;
   common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index a225f9a88..396f711df 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem 
types are @dfn{Amiga
 Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
 @dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
 @dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
 @dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
 @dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
 @dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6267,7 +6268,7 @@ assumed to be encoded in UTF-8.
 NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
 ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
 as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short 
names),
 F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
 to be UTF-8. This might be false on systems configured with legacy charset
 but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 1571421d7..5aaeaef0c 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1438,6 +1438,11 @@ module = {
   common = fs/odc.c;
 };
 
+module = {
+  name = erofs;
+  common = fs/erofs.c;
+};
+
 module = {
   name = ext2;
   common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..de57aaa5e
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,968 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2023 Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABI

[PATCH v4 rebased 2/2] fs/erofs: Add tests for EROFS in grub-fs-tester

2024-02-19 Thread Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.

Signed-off-by: Yifan Zhao 
---
 .gitignore   |  1 +
 Makefile.util.def|  6 ++
 tests/erofs_test.in  | 20 
 tests/util/grub-fs-tester.in | 32 +---
 4 files changed, 52 insertions(+), 7 deletions(-)
 create mode 100644 tests/erofs_test.in

diff --git a/.gitignore b/.gitignore
index 4d0dfb700..9e8730471 100644
--- a/.gitignore
+++ b/.gitignore
@@ -105,6 +105,7 @@ widthspec.bin
 /docs/version-dev.texi
 /docs/version.texi
 /ehci_test
+/erofs_test
 /example_grub_script_test
 /example_scripted_test
 /example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
   dependencies = 'garbage-gen$(BUILD_EXEEXT)';
 };
 
+script = {
+  testcase = native;
+  name = erofs_test;
+  common = tests/erofs_test.in;
+};
+
 script = {
   testcase = native;
   name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+  EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+   exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+   echo "mkfs.erofs not installed; cannot test erofs."
+   exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" 
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+   x"erofs_"*)
+   MINBLKSIZE=4096
+   MAXBLKSIZE=4096
+   ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE" 
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
-   # FS LIMITATION: reiserfs, extN and jfs label is at most 16 
UTF-8 characters
-   x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* | 
x"mdraid"* | x"jfs" | x"jfs_caseins")
+   # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most 
16 UTF-8 characters
+   x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* | 
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or 
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE" 
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
-   | xrockridge_joliet_1999)
+   | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE" 
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
-   x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge | 
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 | 
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+   x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge | 
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 | 
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"* | x"erofs_"*)
INSTDEVICE=/dev/null;;
x"reiserfs")
"mkfs.reiserfs" --format=3.6 -b $BLKSIZE -l "$FSLABEL" -q 
"${MOUNTDEVICE}" ;;
@@ -990,7 +994,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE" 
1); do
x"zfs"*)
OSDIR="grub fs/"
GRUBDIR="($GRUBDEVICE)/grub fs@";;
-   x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge | 
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 | 
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"* | xafs)
+   x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge | 
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_