Hi, i would like to discuss about NUMA API and its short comings when
it comes to memory hierarchy (from fast HBM, to slower persistent
memory through regular memory) and also device memory (which can have
its own hierarchy).
I have proposed a patch to add a new memory topology model to the
kernel for application to be able to get that informations, it
also included a set of new API to bind/migrate process range [1].
Note that this model also support device memory.
So far device memory support is achieve through device specific ioctl
and this forbid some scenario like device memory interleaving accross
multiple devices for a range. It also make the whole userspace more
complex as program have to mix and match multiple device specific API
on top of NUMA API.
While memory hierarchy can be more or less expose through the existing
NUMA API by creating node for non-regular memory [2], i do not see this
as a satisfying solution. Moreover such scheme does not work for device
memory that might not even be accessible by CPUs.
Hence i would like to discuss few points:
- What proof people wants to see this as problem we need to solve ?
- How to build concensus to move forward on this ?
- What kind of syscall API people would like to see ?
People to discuss this topic:
Dan Williams <[email protected]>
Dave Hansen <[email protected]>
Felix Kuehling <[email protected]>
John Hubbard <[email protected]>
Jonathan Cameron <[email protected]>
Keith Busch <[email protected]>
Mel Gorman <[email protected]>
Michal Hocko <[email protected]>
Paul Blinzer <[email protected]>
Probably others, sorry if i miss anyone from previous discussions.
Cheers,
Jérôme
[1] https://lkml.org/lkml/2018/12/3/1072
[2] https://lkml.org/lkml/2018/12/10/1112
