I'm working on the OpenSolaris iSCSI Extensions for RDMA (iSER) project: http://www.opensolaris.org/os/project/iser/
As part of our long range goal of implementing an iSER initiator and target implementation over Infiniband we are also implementing an iSCSI transport for the Opensolaris "Common Multiprotocol SCSI Target" (COMSTAR) framework. More information on COMSTAR, including an architectural block diagram is here: http://www.opensolaris.org/os/project/comstar/ ... and I'll provide a more detailed overview below. This new iSCSI target implementation (in combination with other COMSTAR components) is ultimately intended to supercede the current iSCSI target implemenation (svc:/system/iscsitgt:default) although it may not do so immediately. We have a functional iSCSI prototype implementation that is working well but there are certain design decisions for which we could really use some input from the OpenSolaris networking community. Background on the COMSTAR iSCSI target: COMSTAR is a framework for providing kernel SCSI target services. In addition to the core framework, called "SCSI Target Mode Framework" or STMF, there are two types of add-in modules: lun providers and port providers. A lun provider implements a particular class of SCSI device, like a tape, disk or object storage device. The current implementation in Nevada has one lun provider called "SBD" which allows the export of SCSI block class devices (disks). Port providers implement the physical transport to the SCSI login units defined within the lun provider. The current COMSTAR implementation has one port provider "fct" that enables Fibre Channel transport over certain support HBAs. Our project will introduce a software iSCSI port provider that will allow iSCSI access to the COMSTAR login units in addition to the current Fibre Channel access. Ultimately we will provide an iSER/Infiniband capability as well, and that should be easy to leverage into an iSER/iWarp capability once the RDMA Offload Framework project (http://www.opensolaris.org/os/project/rof/) is complete. Except for the management libraries and CLI utilities, all the COMSTAR components are in the kernel. By extension one of the requirements for our project is to handle the iSCSI data in kernel space so we can package up iSCSI-encapsulated SCSI commands and deliver them to the COMSTAR framework. So... on to the questions: 1. We are currently using the sockfs interface in the kernel to perform our various network tasks, similar to the approach used by the Solaris iSCSI initiator and CIFS server. New connections are handled by soaccept and then we use sosendmsg/sorecvmsg to transfer data on the new iSCSI connection. It's been suggested to me that it would be better to accept and manage connections in user-space. Realizing that ultimately we need to be able to use the resulting connection in the kernel I'd like to understand more about this. Is it possible to accept a connection in user-space and then operate on that connection in a kernel driver? This would require us to add an associated user-space daemon that we don't currently have but I'm not opposed to that if it's the correct way to handle things. If this is a viable approach, what are the advantages? 2. From a networking and/or security standpoint does it make sense to have a discrete SMF service associated with this new COMSTAR iSCSI port provider? The COMSTAR facility itself has an associated SMF service (svc:system/device/stmf) that enables all COMSTAR components at once. So as things are implemented with our prototype today, if the user enables COMSTAR with "svcadm enable stmf" we will immediately start listening for connections on any TCP/IP ports which have been configured for iSCSI (default 3260). The same operation enables any other COMSTAR transports like Fibre Channel. Although this seems to satisfy a basic requirement that network services are disabled by default, it occurred to me that it is not at all clear to users that enabling STMF may conceivably also start up a network service at the same time. Would it be appropriate to have a separate SMF service specifically for iSCSI called svc:/network/iscsi/target:default? I'd appreciate any input that the networking community can provide here. If you have any questions about our implementation that I haven't touched on by all means, please ask. Design documents are on the iSER project page I linked to above and we have a webrev for our current source here: http://cr.opensolaris.org/~pdunlap/iscsit-nwsnv/webrev/ If you are interested in following or participating in the iSER development feel free to subscribe to our development alias: http://mail.opensolaris.org/mailman/listinfo/iser-dev -Peter Dunlap _______________________________________________ networking-discuss mailing list [email protected]
