Hi, On Thu, 20 Apr 2023 at 20:02:27 +0200, Cyril Brulebois wrote: >> * Backport upstream MR !498, let it mature in sid for a few >> weeks then upload 2:2.6.1-4~deb12u1 via t-p-u. There are only 2 >> upstream commits to cherry-pick and neither is large nor intrusive; >> moreover like the commits previously cherry-picked they are no-op on >> “normal” systems (only systems without swap are affected). For >> convenience I attach a debdiff for 2:2.6.1-3~deb12u2 and you'll also >> find binary packages for amd64 at >> https://people.debian.org/~guilhem/tmp/cryptsetup_2.6.1-3~deb12u2/ >> Tested: autopkgtests (incl. full upstream test suite), d-i in both >> graphical and text install on VMs with 1024M RAM (now memory cost >> won't exceed ~250M resp. ~300M thus leaving plenty of headroom for >> the rest). > > Since you're happy with that approach, let's go for an upload to > unstable for the time being, I'll conduct some tests shortly, and once > it's indeed confirmed to work fine, go via t-p-u (because of the same > fun as before with some library) so that it can be used for rc3 (if it's > ready by then — we haven't really defined when it's going to happen > besides “somewhen before end of April”).
Just uploaded 2:2.6.1-4 to sid, and locally prepared a rebuild for
bookworm (2:2.6.1-4~deb12u1).
Comparing PBKDF benchmark results obtained using default settings
(guided “encrypted LVM” partitioning scheme) between the last 3 releases
and 1, 2, or 4G RAM (the first luksDump is what I got out of d-i, the
second shows benchmark results on the final system — with swap), I get
the following parameters (summary at the bottom).
Buster (debian-10.12.0-amd64-netinst.iso, text install), 1024M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 504962
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 8
Memory: 505350
Threads: 2
Buster (debian-10.12.0-amd64-netinst.iso, text install), 2048M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 538914
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 1021446
Threads: 2
Buster (debian-10.12.0-amd64-netinst.iso, text install), 4096M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 533886
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 1048576
Threads: 2
Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 1024M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 499892
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 8
Memory: 499888
Threads: 2
Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 2048M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 582804
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 1015216
Threads: 2
Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 4096M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 518981
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2i
Time cost: 4
Memory: 948373
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 1024M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 5
Memory: 489820
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 8
Memory: 490598
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 2048M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 553835
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 1005926
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 4096M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 546642
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 1048576
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup
2:2.6.1-4~deb12u1,
graphical install), 1024M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 10
Memory: 223780
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 8
Memory: 490598
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup
2:2.6.1-4~deb12u1,
text install), 1024M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 8
Memory: 294302
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 8
Memory: 490598
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup
2:2.6.1-4~deb12u1,
text install), 2048M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 590553
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 1005926
Threads: 2
Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup
2:2.6.1-4~deb12u1,
text install), 4096M RAM:
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 613826
Threads: 2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test
root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF: argon2id
Time cost: 4
Memory: 1048576
Threads: 2
Bottom line:
* The upstream patches in the patch-queue (the 2 backported earlier
from upstream MR !490 plus the new other two from upstream MR !498)
only affect systems with <2G RAM (i.e., those where half the amount
of physical memory is lower than DEFAULT_LUKS2_MEMORY_KB). And only
those without swap. On such systems the memory cost is set to a
lower value at the expense of a higher time cost, which is the
intended behavior; it appear to leave enough head-room for the
graphical installer to succeed with 1G RAM, so I believe the errata
can be removed if the changes makes it to bookworm.
* I was surprised to see the memory cost settle at ~550-600M on systems
with a decent amount of RAM in d-i. Would have expected to see 1G
here just like after running `cryptsetup luksConvertKey` in the
normal system. I get a similarily low memory cost after dropping to
a rescue shell early in d-i and running `luksFormat` manually:
~ # grep -c ^processor /proc/cpuinfo
6
~ # free
total used free shared buff/cache
available
Mem: 6062584 107888 5647804 260000 306892
5543168
Swap: 0 0 0
~ # echo test | cryptsetup luksFormat --debug --batch-mode /dev/sda
[…]
# Running argon2id() benchmark.
# PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4
(took 229 ms)
# PBKDF benchmark: memory cost = 71545, iterations = 4, threads = 4
(took 242 ms)
# PBKDF benchmark: memory cost = 73910, iterations = 4, threads = 4
(took 249 ms)
# PBKDF benchmark: memory cost = 74206, iterations = 4, threads = 4
(took 246 ms)
# PBKDF benchmark: memory cost = 75412, iterations = 4, threads = 4
(took 254 ms)
# PBKDF benchmark: memory cost = 593795, iterations = 4, threads = 4
(took 3527 ms)
# PBKDF benchmark: memory cost = 336713, iterations = 4, threads = 4
(took 1196 ms)
# PBKDF benchmark: memory cost = 563065, iterations = 4, threads = 4
(took 2035 ms)
# Benchmark returns argon2id() 4 iterations, 563065 memory, 4 threads
(for 512-bits key).
[…]
I think what happens here is that compared to the final system d-i is
a bit crippled so the 2s threshold is reached earlier in the
benchmark. For comparison, running the benchmark in the initramfs
shell of the final system (after installation, but also without
swap):
(initramfs) free
total used free shared buff/cache
available
Mem: 6064140 66752 5797144 56 200244
5675728
Swap: 0 0 0
(initramfs) echo test | cryptsetup luksConvertKey --debug --batch-mode
/dev/sda5
[…]
# Running argon2id() benchmark.
# PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4
(took 94 ms)
# PBKDF benchmark: memory cost = 174297, iterations = 4, threads = 4
(took 239 ms)
# PBKDF benchmark: memory cost = 182319, iterations = 4, threads = 4
(took 242 ms)
# PBKDF benchmark: memory cost = 188346, iterations = 4, threads = 4
(took 243 ms)
# PBKDF benchmark: memory cost = 193771, iterations = 4, threads = 4
(took 232 ms)
# PBKDF benchmark: memory cost = 208804, iterations = 4, threads = 4
(took 274 ms)
# PBKDF benchmark: memory cost = 1048576, iterations = 5, threads = 4
(took 1721 ms)
# Benchmark returns argon2id() 5 iterations, 1048576 memory, 4 threads
(for 512-bits key).
[…]
And now in the final system fully booted (same result as in initramfs):
root@debian:~# free -h
total used free shared buff/cache
available
Mem: 5.8Gi 270Mi 5.6Gi 476Ki 78Mi
5.5Gi
Swap: 975Mi 0B 975Mi
root@debian:~# cryptsetup luksConvertKey --debug --batch-mode /dev/sda5
<<<test
[…]
# Running argon2id() benchmark.
# PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4
(took 93 ms)
# PBKDF benchmark: memory cost = 176172, iterations = 4, threads = 4
(took 248 ms)
# PBKDF benchmark: memory cost = 177592, iterations = 4, threads = 4
(took 242 ms)
# PBKDF benchmark: memory cost = 183462, iterations = 4, threads = 4
(took 226 ms)
# PBKDF benchmark: memory cost = 202944, iterations = 4, threads = 4
(took 274 ms)
# PBKDF benchmark: memory cost = 1048576, iterations = 5, threads = 4
(took 1795 ms)
# Benchmark returns argon2id() 5 iterations, 1048576 memory, 4 threads
(for 512-bits key).
[…]
Never noticed that before, but that's not a regression since buster
and bullseye both have the same behavior. (At least in my test VMs;
didn't compare on real hardware.)
Cheers
--
Guilhem.
signature.asc
Description: PGP signature
