From: John Groves <j...@groves.net> This patch set is not intended to be merged; I'm hoping to get some clarification as to the correct approach (especialy from Dan).
This work is related to famfs, which is a dax file system for shared fabric-attached memory (FAM). Famfs is "coming soon" as an RFC, but the concept and requirements were presented at LPC 2023. See https://lpc.events/event/17/contributions/1455/ and https://www.youtube.com/watch?v=aA_DgO95gLo. My expectation is that a future (fully working) version of this patch will be folded into the famfs patches. Unlike the current set of fs-dax file systems, famfs does not need a block (pmem) device, and should really run on a /dev/dax character device since that's how sharable fabric-attached cxl memory will surface. But /dev/dax character devices are missing some functionality that is provided by the block /dev/pmem driver - specifically struct dax_operations pointer in struct dax_device. This patch, when CONFIG_DEV_DAX_IOMAP=y, populates dax_dev->ops for character dax devices. The added operations differ (currently) from /dev/pmem's dev_dax->ops in that they don't use memremap but instead provide a physical address in response to the dev_dax->direct_access() method. The dax_operations are direct_access() (which resolves a dax dev offset to an address), zero_page_range() and recovery_write(). I'm not sure yet how to test the latter two, but the direct_access() method works in conjunciton with famfs - but only for mmaped files. But Posix reads fail. Specifically dax_iomap_iter() calls dax_copy_to_iter(), which declines to copy the data for some reason in one of the lower level copy_to_user variants. I've tried to isolate the reason for the failure with a VM under gdb, but no luck there yet. I'm wondering if there is some flag or attribute that needs to be applied to these addresses/pages somehow to allow this to work. The failing copy_to_user() differs from the same path with pmem fs-dax, in that pmem does a memremap (which I think generates one contiguous range, even if the device had more than one range - is this right, and does this mean it's consuming some of the vmap/vmalloc range?) I spent some time attempting a memremap, but I haven't figured out the magic for that. However, I like the simplicity of resolving to phys if that's not a non-starter for some reason. I hope this is enough context for a meaningful review and suggestions as to what a working dev_dax->dax_operations implementation should look like. Thanks for any tips! John Groves (4): Add add_dax_ops() func for fs-dax to provide dax holder_ops Temporary hacks due to linkage issues Add dax_operations to /dev/dax struct dax_device Add CONFIG_DEV_DAX_IOMAP kernel build parameter drivers/dax/Kconfig | 6 ++ drivers/dax/bus.c | 155 ++++++++++++++++++++++++++++++++++++++++++++ drivers/dax/super.c | 16 +++++ include/linux/dax.h | 5 ++ 4 files changed, 182 insertions(+) -- 2.40.1
