Dan, thank you very much for the reply. I appreciate your time on this.
I talked to my application developer and he said that he can work around the
lack of coherency that you identified.
I was under the impression that PCIe memory physical ranges were always mapped
as UC which obviously implies that they are not cacheable but I guess I am
wrong there.
With that said, I still would like to create a /dev/daxX.Y character device
that would map to the PCIe BAR range.
I am using driver/dax/device.c as my prototype.
a) I added memmap=X@Y to the kernel command line. When I rebooted the kernel, I
see /dev/pmem0. So far so good.
Ndctl list shows this memory with namespace0.0
b) I then ran ndctl create namespace with --mode=devdax to convert from fsdax
to devdax and noticed that the probe() routine in device.c was invoked as I
expected.
Now in my driver, I only have my PCIe function probe() routine. I was wondering
if there is an obvious way for me to make this work just like PMEM (memmap).
There is a lot of functionality provided in other source files of drivers/dax
and it all seems to depend on having that namespace created and lots of
initialization being done.
I was wondering if there is an easier way for me to attach a struct dev_dax to
my own pcie functionality. It does not look like there is an easy way but
wanted to check with you
-----Original Message-----
From: Dan Williams <[email protected]>
Sent: Tuesday, April 26, 2022 8:55 PM
To: An Sarpal <[email protected]>
Cc: [email protected]
Subject: Re: How to map my PCIe memory as a devdax device (/dev/daxX.Y)
On Tue, Apr 26, 2022 at 5:31 PM An Sarpal <[email protected]> wrote:
>
> Dan, thank you.
>
> Yes that is true but PCIe memory (assuming a PCIe BAR is backed by real
> memory and not just registers) never participated in cache coherence anyways.
> So assuming my applications using /dev/dax0.0 character device were aware of
> this feature (or lack of it), would this implementation be the correct way to
> do it?
It does not matter if the BAR is backed by real memory. It is incoherent with
other initiators on the bus, so it can create cases where dirty data is
stranded in CPU caches while a another bus initiator tries access the same
physical address.
