Hi all,
I’m currently doing some load tests/burn in for two new servers.
These feature all NVME SSDs and run FreeNAS, i.e. FreeBSD 11.2-STABLE.
pcib17: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 1 on pci15
pcib17: [GIANT-LOCKED]
pci17: <ACPI PCI bus> numa-domain 1 on pcib17
nvme7: <Generic NVMe Device> mem 0xeca10000-0xeca13fff at device 0.0
numa-domain 1 on pci17
When putting some moderate i/o load on the system, the log fills with these
messages:
nvme7: aborting outstanding i/o
nvme7: DATASET MANAGEMENT sqid:41 cid:91 nsid:1
nvme7: ABORTED - BY REQUEST (00/07) sqid:41 cid:91 cdw0:0
There has been some discussion of this on on the iX Systems forum as well as
various
FreeBSD media and one person suggested setting:
hw.nvme.per_cpu_io_queues=0
This is where I need some help now. This is from the manpage for nvme(4):
----------
To force a single I/O queue pair shared by all CPUs, set the following
tunable value in loader.conf(5):
hw.nvme.per_cpu_io_queues=0
To assign more than one CPU per I/O queue pair, thereby reducing the
number of MSI-X vectors consumed by the device, set the following tunable
value in loader.conf(5):
hw.nvme.min_cpus_per_ioq=X
To force legacy interrupts for all nvme driver instances, set the
following tunable value in loader.conf(5):
hw.nvme.force_intx=1
Note that use of INTx implies disabling of per-CPU I/O queue pairs.
----------
But:
root@freenas01[~]# sysctl hw.nvme.per_cpu_io_queues
sysctl: unknown oid 'hw.nvme.per_cpu_io_queues'
root@freenas01[~]# sysctl hw.nvme.min_cpus_per_ioq
sysctl: unknown oid 'hw.nvme.min_cpus_per_ioq'
root@freenas01[~]# sysctl hw.nvme.force_intx
sysctl: unknown oid 'hw.nvme.force_intx'
Where do I go from here?
Thanks!
Patrick
--
punkt.de GmbH Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe i...@punkt.de http://punkt.de
AG Mannheim 108285 Gf: Juergen Egeling
_______________________________________________
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
