This patch series speeds up LUKS PBKDF2 unlocking by using assembly files from libgcrypt 1.11.0 to enable hardware-accelerated SHA-256 and SHA-512.
As an initial step, the scope is limited to x86_64 EFI targets. Patch 1 modifies autoconf/automake files to check the availablity of special CPU instructions. Patch 2 and 3 introduce the functions to enable/disable the hardware features. Patch 4 and 5 copy the selected assembly files. Patch 6 implements _gcry_get_hw_features() to control the hardware features detection in libgcrypt. Patch 7 fixes a potential build error when compiling the Intel SHA extention function for the gcry_sha256 module. Patch 8 and 9 build the gcry_sha256 and gcry_sha512 modules with the assembly files. Patch 10 adds a new option to cryptomount to enable hardware acceleration. NOTE: The libgcrypt patch numbering begins at 13 to follow the Argon2 patch set(*). (*) https://lists.gnu.org/archive/html/grub-devel/2025-10/msg00235.html v2: - Fixing the coding style issues in patch 3 - Removing 'inline' from get_cpuid_ecx() and get_cpuid_ecx() - Rebasing to the latest git master Gary Lin (10): Tweak autoconf/automake files to detect x86_64 features lib/hwfeatures-gcry: Introduce functions to manage hardware features lib/hwfeatures-gcry: Enable SSE and AVX for x86_64 EFI libgcrypt: Copy sha256 x86_64 assembly files libgcrypt: Copy sha512 x86_64 assembly files libgcrypt: Implement _gcry_get_hw_features() libgcrypt: Declare the sha256 shaext function libgcrypt: Add hardware acceleration for gcry_sha256 libgcrypt: Add hardware acceleration for gcry_sha512 disk/cryptodisk: Add '--hw-accel' to enable hardware acceleration Makefile.util.def | 1 + autogen.sh | 11 + conf/Makefile.common | 2 + conf/Makefile.extra-dist | 4 + configure.ac | 233 +++++++++++++++++ docs/grub.texi | 5 +- grub-core/Makefile.core.def | 2 + grub-core/disk/cryptodisk.c | 26 +- grub-core/lib/hwfeatures-gcry.c | 52 ++++ .../libgcrypt-patches/13_add_hwfeatures.patch | 87 +++++++ .../14_fix_build_shaext.patch | 35 +++ .../15_build_sha256_x86_64_efi_opt_code.patch | 43 +++ .../16_build_sha512_x86_64_efi_opt_code.patch | 35 +++ grub-core/lib/x86_64/efi/hwfeatures-gcry.c | 246 ++++++++++++++++++ grub-core/normal/main.c | 3 +- include/grub/crypto.h | 7 +- include/grub/hwfeatures-gcry.h | 26 ++ include/grub/x86_64/cpuid.h | 1 + include/grub/x86_64/efi/hwfeatures-gcry.h | 25 ++ 19 files changed, 836 insertions(+), 8 deletions(-) create mode 100644 grub-core/lib/hwfeatures-gcry.c create mode 100644 grub-core/lib/libgcrypt-patches/13_add_hwfeatures.patch create mode 100644 grub-core/lib/libgcrypt-patches/14_fix_build_shaext.patch create mode 100644 grub-core/lib/libgcrypt-patches/15_build_sha256_x86_64_efi_opt_code.patch create mode 100644 grub-core/lib/libgcrypt-patches/16_build_sha512_x86_64_efi_opt_code.patch create mode 100644 grub-core/lib/x86_64/efi/hwfeatures-gcry.c create mode 100644 include/grub/hwfeatures-gcry.h create mode 100644 include/grub/x86_64/cpuid.h create mode 100644 include/grub/x86_64/efi/hwfeatures-gcry.h Range-diff against v1: 1: 4c2b3fec8 < -: --------- util/import_gcry: Import kdf.c for Argon2 2: 210b6292b < -: --------- crypto: Update crypto.h for libgcrypt KDF functions 3: 3b72e07b8 < -: --------- libgcrypt/kdf: Implement blake2b_512.hash_buffers() 4: d90a709b1 < -: --------- libgcrypt/kdf: Get rid of gpg_err_code_from_errno() 5: 5980e8a1a < -: --------- libgcrypt/kdf: Remove unsupported KDFs 6: 6f3a415ca < -: --------- libgcrypt/kdf: Fix 64-bit modulus on 32-bit platforms 7: 8298d3139 < -: --------- argon2: Introduce grub_crypto_argon2() 8: e3ff82e7c < -: --------- Import Argon2 tests from libgcrypt 9: 168a8d24b < -: --------- Integrate Argon2 tests into functional_test 10: 3bc593992 < -: --------- disk/luks2: Add Argon2 support 11: 1fb5cac29 < -: --------- tests/util/grub-fs-tester: Use Argon2id for LUKS2 test 12: 840cf2fb0 < -: --------- docs: Document argon2 and argon2_test 13: 1fbd2a278 < -: --------- kern/misc: Implement faster grub_memcpy() for aligned buffers 14: 3b2ccefb0 ! 1: 1890cf937 Tweak autoconf/automake files to detect x86_64 features @@ Commit message detection is currently limited to the x86_64 EFI target. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## conf/Makefile.common ## @@ conf/Makefile.common: if COND_HAVE_PCI 15: e92856ba5 ! 2: 0992871e9 lib/hwfeatures-gcry: Introduce functions to manage hardware features @@ Commit message functions and sets '__gcry_use_hwf' to 'false'. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## Makefile.util.def ## @@ Makefile.util.def: library = { 16: 43f2e43de ! 3: baf25b5ad lib/hwfeatures-gcry: Enable SSE and AVX for x86_64 EFI @@ Commit message on x86_64 EFI systems when the hardware is capable. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## grub-core/Makefile.core.def ## @@ grub-core/Makefile.core.def: module = { @@ grub-core/lib/x86_64/efi/hwfeatures-gcry.c (new) + * + * Ref: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=aa5cacdc29d76a005cbbee018a47faa6e724dd2d + */ -+#define __FORCE_ORDER "m"(*(unsigned int *)0x1000UL) ++#define __FORCE_ORDER "m"(*(unsigned int *) 0x1000UL) + +#define HW_FEATURE_X86_64_SSE (1 << 0) +#define HW_FEATURE_X86_64_AVX (1 << 1) @@ grub-core/lib/x86_64/efi/hwfeatures-gcry.c (new) +read_cr0 (void) +{ + grub_uint64_t val; ++ + asm volatile ("mov %%cr0, %0" : "=r" (val) : __FORCE_ORDER); + return val; +} @@ grub-core/lib/x86_64/efi/hwfeatures-gcry.c (new) +read_cr4 (void) +{ + grub_uint64_t val; ++ + asm volatile ("mov %%cr4,%0" : "=r" (val) : __FORCE_ORDER); + return val; +} @@ grub-core/lib/x86_64/efi/hwfeatures-gcry.c (new) + asm volatile ("mov %0,%%cr4": "+r" (val) : : "memory"); +} + -+static inline grub_uint32_t ++static grub_uint32_t +get_cpuid_ecx (void) +{ + grub_uint32_t eax, ebx, ecx, edx; @@ grub-core/lib/x86_64/efi/hwfeatures-gcry.c (new) + return ecx; +} + -+static inline grub_uint32_t ++static grub_uint32_t +get_cpuid_edx (void) +{ + grub_uint32_t eax, ebx, ecx, edx; 17: fa139ede3 ! 4: 5195b898a libgcrypt: Copy sha256 x86_64 assembly files @@ Commit message acceleration for sha256. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## autogen.sh ## @@ autogen.sh: for x in mpi-asm-defs.h mpih-add1.c mpih-sub1.c mpih-mul1.c mpih-mul2.c mpih-mul 18: 7cf761b87 ! 5: 77d309e18 libgcrypt: Copy sha512 x86_64 assembly files @@ Commit message acceleration for sha512. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## autogen.sh ## @@ autogen.sh: for x in mpi-asm-defs.h mpih-add1.c mpih-sub1.c mpih-mul1.c mpih-mul2.c mpih-mul 19: 09f2baf1e ! 6: 61042e067 libgcrypt: Implement _gcry_get_hw_features() @@ Commit message for x86_64. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## autogen.sh ## @@ autogen.sh: for x in sha256-ssse3-amd64.S sha256-avx-amd64.S sha256-avx2-bmi2-amd64.S sha256 20: 9176b4266 ! 7: 9a5776f31 libgcrypt: Declare the sha256 shaext function @@ Commit message Declare the prototype in sha256-intel-shaext.c to avoid the error. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## conf/Makefile.extra-dist ## @@ conf/Makefile.extra-dist: EXTRA_DIST += grub-core/lib/libgcrypt-patches/10-kdf-use-GPG-errs.patch 21: 0e1265bf8 ! 8: 2df17f531 libgcrypt: Add hardware acceleration for gcry_sha256 @@ Commit message for the x86_64 EFI target. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## conf/Makefile.extra-dist ## @@ conf/Makefile.extra-dist: EXTRA_DIST += grub-core/lib/libgcrypt-patches/11-kdf-remove-unsupported-kdfs.pat 22: d6f0e1707 ! 9: ba3a8ac3a libgcrypt: Add hardware acceleration for gcry_sha512 @@ Commit message for the x86_64 EFI target. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## conf/Makefile.extra-dist ## @@ conf/Makefile.extra-dist: EXTRA_DIST += grub-core/lib/libgcrypt-patches/12-kdf-use-grub_divmod64.patch 23: ab712f1fd ! 10: 9798849dd disk/cryptodisk: Add '--hw-accel' to enable hardware acceleration @@ Commit message availability of the new option. Signed-off-by: Gary Lin <[email protected]> + Reviewed-by: Daniel Kiper <[email protected]> ## docs/grub.texi ## @@ docs/grub.texi: The option @option{--quiet} can be given to suppress the output. @@ grub-core/normal/main.c: static const char *features[] = { "feature_default_font_path", "feature_all_video_module", "feature_menuentry_id", "feature_menuentry_options", "feature_200_final", "feature_nativedisk_cmd", "feature_timeout_style", -- "feature_search_cryptodisk_only" -+ "feature_search_cryptodisk_only", "feature_gcry_hw_accel" +- "feature_search_cryptodisk_only", "feature_tpm2_cap_pcrs" ++ "feature_search_cryptodisk_only", "feature_tpm2_cap_pcrs", ++ "feature_gcry_hw_accel" }; GRUB_MOD_INIT(normal) -- 2.51.0 _______________________________________________ Grub-devel mailing list [email protected] https://lists.gnu.org/mailman/listinfo/grub-devel
