On 22/09/2026 16:07, Milan Broz wrote:
> Could you please point me to at least one consumer-grade NVMe drive that
> supports a 64k sector size?
None that I know of, and none is needed - I think my cover letter invited
that reading, sorry.
The encryption sector size is dm-crypt's crypto chunking unit, not a
property required of the backing device. dm-crypt never validates it
against the device. The only use of bdev_logical_block_size() in the
target is
sector_align = max(bdev_logical_block_size(cc->dev->bdev),
(unsigned)cc->sector_size);
in get_max_request_sectors(), which is request alignment, not a check.
A mapping whose encryption sector exceeds the drive's block size is
already the normal case - sector_size:4096 on a 512e drive is exactly
that.
The drive is also not where the gain comes from; see the last point.
> Allowing a bigger sector size in dm-crypt means you can end up with
> partial dm-crypt sector writes (after power fail), which opens another
> can of worms.
This is the objection I take most seriously, and you are right that it
is real.
It is not a new hazard though - it is the existing one scaled. The same
tearing is possible whenever the encryption sector exceeds the drive's
atomic unit, which is every sector_size:4096 mapping on a 512e drive.
cryptsetup already documents precisely this for --sector-size:
"Note that using a sector size larger than the underlying storage
device's physical sector size may result in data corruption during
unexpected power failures. A power failure during write operations
may result in only partial completion of the encryption sector
write, leaving encrypted data in an inconsistent state that cannot
be properly decrypted."
What a larger sector changes is the width of the window, not its
existence, and only for a mapping that asks for it - the default stays
512 bytes.
For the common case the failure mode is also unchanged: with xts(aes)
each 16-byte block carries its own tweak, so a torn write leaves a
mixture of old and new plaintext at that granularity rather than
destroying the sector. The modes that would be worse are already
restricted - crypt_iv_lmk_ctr() and crypt_iv_tcw_ctr() reject anything
but 512 bytes. The exception I am aware of is the Elephant diffuser,
which does diffuse across the whole sector; it is reachable only by
hand-writing a table, since cryptsetup's BITLK support allows 512 and
4096 only, but I mention it rather than leave you to find it.
I would rather document this than leave it implicit. If the patch is
worth pursuing at all, I will add to
Documentation/admin-guide/device-mapper/dm-crypt.rst:
An encryption sector larger than the atomic write unit of the
underlying device can be torn by a power failure, leaving part of
the sector written and part not. This is already possible with a
4096 byte sector on a 512e device; a larger sector widens the
window. Use one only where that is acceptable.
Reworded however you prefer.
> Your dm-verity argument does not apply here; it is a read-only target.
You are right, and I overreached. dm-verity shows only that PAGE_SIZE is
an accepted bound for how much data one target request may cover; it
says nothing about write atomicity, which is the part that matters here.
I will drop the comparison.
> Also, arguments based on cryptsetup/LUKS2 do not make much sense for
> kernel code. They must be compatible and work together, but dm-crypt
> can be used without any userspace validation, so all limits must be
> checked in the kernel.
Agreed, and that is what the patch does. The limit is enforced in
crypt_ctr_optional():
cc->sector_size > DM_CRYPT_MAX_SECTOR_SIZE
with DM_CRYPT_MAX_SECTOR_SIZE = min(PAGE_SIZE, BLK_MAX_BLOCK_SIZE). It
needs no userspace cooperation: dmsetup and a raw ioctl are bound by it
exactly as cryptsetup is, and on a 4k-page kernel it is still 4096, so
nothing changes there at all.
The LUKS2 4k rule is a different kind of thing. It is a property of an
on-disk format the kernel knows nothing about, so it cannot be enforced
here and I am not proposing that it should be - it stays in cryptsetup,
untouched. I raised it only to answer the portability objection from the
v1 thread, not as an argument for kernel behaviour. Ondrej made the same
point back to me on the cryptsetup list earlier today, and he was right:
https://lore.kernel.org/cryptsetup/[email protected]/
> LUKS2 limits the sector size to 4k for multiplatform compatibility.
Yes, and nothing in this patch changes that.
> dm-crypt itself can support bigger sectors, but I just do not see much
> use for it with common NVMe drives or other storage.
Storage is not the use case - crypto offload is.
Where the cipher is a hardware engine driven over DMA, the per-request
cost is a descriptor setup and a round trip, and that cost dominates.
Making the request sixteen times larger amortises it. With the in-tree
qce driver on an arm64 64k-page board, plain dm-crypt over a ramdisk,
MB/s:
sector_size seq read rand read seq write rand write
4096 13.3 22.8 27.2 24
65536 581 582 586 588
The ramdisk is deliberate - it keeps the storage out of the measurement,
because the storage is not what is being fixed.
That also answers the NVMe half of the question, and I should have made
it the main point rather than the ramdisk. A common consumer NVMe drive
delivers something in the region of 1-3 GB/s. Driving qce with a
4096-byte sector, dm-crypt manages 13-27 MB/s, so behind any such drive
the crypto engine is the limit rather than the drive, by about two
orders of magnitude. At 65536 bytes it reaches roughly 580 MB/s and the
engine is still the limit. The ramdisk figures are therefore a
reasonable predictor of what the same setup does on ordinary storage:
nothing unusual is being asked of the drive, only that the cipher is
offloaded to an engine reached over DMA. The drive can be entirely
common - it is the accelerator behind it that is currently being driven
inefficiently.
The other measurement in the cover letter is the NVMe one, and it is a
different board: an arm64 Cortex-A53, also 64k pages, but with the CPU
cipher xts-aes-ce over a Samsung 970 EVO. There the gain is 9-16%,
which matches your intuition - for a CPU cipher there is very little in
this, and the drive is not the limit either.
So I would put the case as: it is not about the drive at all. It is for
systems that have a crypto accelerator and a 64k page granule, where
dm-crypt currently leaves most of the engine's throughput unused - and
the storage behind them can be an entirely ordinary NVMe. If that is
still too narrow to justify the option, that is a fair conclusion to
reach, but it is the case I am asking about, and the numbers are with
an in-tree driver.
Thanks for taking the time on this.
Itai
