Hello,
I am relatively new to Lustre. I have a running Lustre setup and would like to
ensure that there is no cross-numa traffic going on in the server side.
Each Lustre OSS has the following:
* Dual socket AMD CPUs
* 4 NUMA regions (2 NUMA going to each socket)
* 2x Mellanox NIC (one for each CPU socket).
* 16x NVMe drives (8 for each CPU socket).
The way I currently have it set up:
* Each group of 8 NVMe on the same socket, I created a RAID0 group. Each
RAID0 group is an OST.
* In the LNET configuration, each NIC is assigned to the corresponding CPT.
We are using RDMA over Ethernet.
* o2ib0(ens3f0np0)[0,1], o2ib1(ens7f0np0)[2,3]
According to the documentation, messages coming to the OSS will have the
corresponding CPT/NUMA cores that match the NIC handle the incoming traffic.
However when accessing the OST themselves I did not find any tuning to bind a
specific OST to CPU cores.
I did see an option called “options ksocklnd enable_irq_affinity=0” but it
appears it is only for the ethernet driver and not RDMA.
Is there a way to limit each OST’s read/write threads to a specific set of CPU
cores?
Thank you,
Jeff
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
