Module Name: src
Committed By: jdolecek
Date: Wed Sep 21 20:01:03 UTC 2016
Modified Files:
src/share/man/man4: nvme.4
Log Message:
expand driver description
To generate a diff of this commit:
cvs rdiff -u -r1.4 -r1.5 src/share/man/man4/nvme.4
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/share/man/man4/nvme.4
diff -u src/share/man/man4/nvme.4:1.4 src/share/man/man4/nvme.4:1.5
--- src/share/man/man4/nvme.4:1.4 Fri Sep 16 12:43:37 2016
+++ src/share/man/man4/nvme.4 Wed Sep 21 20:01:03 2016
@@ -1,4 +1,4 @@
-.\" $NetBSD: nvme.4,v 1.4 2016/09/16 12:43:37 wiz Exp $
+.\" $NetBSD: nvme.4,v 1.5 2016/09/21 20:01:03 jdolecek Exp $
.\" $OpenBSD: nvme.4,v 1.2 2016/04/14 11:53:37 jmc Exp $
.\"
.\" Copyright (c) 2016 David Gwynne <d...@openbsd.org>
@@ -15,7 +15,7 @@
.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
.\"
-.Dd September 16, 2016
+.Dd September 21, 2016
.Dt NVME 4
.Os
.Sh NAME
@@ -29,10 +29,77 @@ The
driver provides support for NVMe, or NVM Express,
storage controllers conforming to the
Non-Volatile Memory Host Controller Interface specification.
+Controllers complying to specification version 1.1 and 1.2 are known to work.
+Other versions should too for normal operation with exception of some
+passthrough commands.
+.Pp
+Driver supports following features:
+.Bl -bullet -compact -offset indent
+.It
+controller and namespace configuration and management using
+.Xr nvmectl 1
+.It
+highly parallel I/O using per-CPU I/O queues
+.It
+PCI MSI/MSI-X attachment, and INTx for legacy systems
+.El
+.Pp
+On systems supporting MSI/MSI-X,
+.Nm
+driver uses per-CPU IO queue pairs for lockless and highly parallelized I/O.
+Interrupt handlers are scheduled on distinct CPUs.
+Driver allocates as many interrupt vectors as available, up to number
+of CPUs + 1.
+MSI supports up to 32 interrupt vectors within the system,
+MSI-X can have up to 2k.
+Each I/O queue pair has separate command circular buffer.
+.Nm
+specification allows up to 64k commands per queue, driver currently allocates
+1024 items per queue by default.
+Command submissions are done always on current CPU, command completion
+interrupt is handled on CPU according to I/O queue ID - first I/O queue on CPU0,
+second I/O queue on CPU1 etc.
+Admin queue command completion is not tied to any CPU, it's handled by
+any CPU.
+To keep lock contention to minimum, it's recommended to keep this assignment,
+even thought it is possible to reassign the interrupt handlers differently,
+using
+.Xr intrctl 1 .
+Driver also uses soft interrupts to process command completions, in order to
+increase total system I/O capacity and throughput.
+.Pp
+On systems without MSI, driver uses single HW interrupt handler, for
+both admin and standard I/O commands.
+Command submissions are done on current CPU, command completion interrupt
+is handled on any available CPU. This leads to some lock contention,
+especially on command ccbs.
+Also, command completion handling must be done within the HW interrupt
+handler.
+.Sh FILES
+.Bl -tag -width /dev/nvmeX -compact
+.It Pa /dev/nvme*
+nvme device special files used by
+.Xr nvmectl 1 .
+.El
.Sh SEE ALSO
.Xr intro 4 ,
.Xr ld 4 ,
-.Xr pci 4
+.Xr pci 4 ,
+.Xr nvmectl 1 ,
+.Xr MAKEDEV 8 ,
+.Xr intrctl 1
+.Rs
+.%A NVM Express, Inc.
+.%T "NVM Express \- scalable, efficient, and industry standard"
+.%D 2016-06-12
+.%U http://nvmexpress.org/
+.Re
+.Rs
+.%A NVM Express, Inc.
+.%T "NVM Express Revision 1.2.1"
+.%D 2016-06-05
+.%U http://www.nvmexpress.org/wp-content/uploads/NVM_Express_1_2_1_Gold_20160603.pdf
+.Re
.Sh HISTORY
The
.Nm
@@ -54,24 +121,32 @@ and ported to
by
.An NONAKA Kimihiro
.Aq Mt non...@netbsd.org .
-.Sh BUGS
-Emulated
-.Nm
-device under QEMU doesn't currently work - controller identify command times
-out.
+.An Jaromir Dolecek
+.Aq Mt jdole...@netbsd.org
+contributed to making this driver MPSAFE.
+.Sh NOTES
+At least some
+.Tn Intel
+.Nm
+adapter cards are known to require
+.Tn PCIe
+Generation 3 slot. Such cards do not even probe when plugged
+into older generation slot.
.Pp
-Emulated
+Driver attaches and works fine also for emulated
.Nm
-device under
+device under QEMU and
.Tn Oracle
+.Tn VirtualBox .
+However,
+there seems to be some broken interaction between
.Tn VirtualBox
-attaches and works fine, but sometimes after reboot device no longer works.
-When this happens, controller fails to properly disable/enable during probe,
-so the
-.Xr ld 4
-is not attached.
-Virtual machine must be completely powered off and started
-again to get the device working again.
+5.1.6
+and the driver, emulated
+.Nm
+controller responds to commands only first time it's attached, after reboot or
+module reload stops responding. Virtual machine must be completely powered off
+(or even killed) to fix.
.Pp
.Nm
kernel module is currently only loadable for kernels configured with
