Re: [PATCH 00/10] nvme multipath support on top of nvme-4.13 branch
On Fri, Sep 15, 2017 at 08:07:01PM +0200, Christoph Hellwig wrote: > Hi Anish, > > I looked over the code a bit, and I'm rather confused by the newly > added commands. Which controller supports them? Also the NVMe > working group went down a very different way with the ALUA approch, > which uses different grouping concepts and doesn't require path > activations - for Linux we'd really like to stick to only standardized > multipath implementations. Hi Christoph, Thanks for looking over the code. The newly added commands were to support Asymmetric NVME Namespace Subsystem. Though, I can remove those commands and make it active-active configuration. I noticed you have sent out the review with changes adding new structures and functions which makes it very similar in terms of implementation. The only reason I proposed given changes, as it doesn't require any change to kernel.
Re: [PATCH 00/10] nvme multipath support on top of nvme-4.13 branch
On Wed, Sep 13, 2017 at 08:57:13AM +0200, Hannes Reinecke wrote: > In general I am _not_ in favour of this approach. > > This is essentially the same level of multipath support we had in the > old qlogic and lpfc drivers in 2.4/2.6 series, and it took us _years_ to > get rid of this. > Main objection here is that it will be really hard (if not impossible) > to use the same approach for other subsystems (eg SCSI), so we'll end up > having different multipath implementations depending on which subsystem > is being used. > Which really is a maintenance nightmare. > > I'm not averse to having other multipath implementations in the kernel, > but it really should be abstracted so that other subsystems can _try_ to > leverage it. Got it. Thanks!
Re: [PATCH 00/10] nvme multipath support on top of nvme-4.13 branch
Hi Anish, I looked over the code a bit, and I'm rather confused by the newly added commands. Which controller supports them? Also the NVMe working group went down a very different way with the ALUA approch, which uses different grouping concepts and doesn't require path activations - for Linux we'd really like to stick to only standardized multipath implementations.
Re: [PATCH 00/10] nvme multipath support on top of nvme-4.13 branch
On 09/12/2017 06:20 AM, Anish M Jhaveri wrote: > Hello everyone, > Please find patchset for supporting Multipathing using nvme namespace > and nvme controller. > > Basic idea behind the design and implementation is to re-use the same > logic which is between generic nvme namespace and it's controller. > One controller to many namespaces. Similarly, this implementation uses > one multipath controller and sibling namespaces. > It creates a head multipath nvme namespace as soon as it find a nvme > namespace during enumeration which has property of being shared. Prior > to creating new head multipath device, nvme namespace check for match- > ing NGUID, if it finds one, this nvme namespace is added as a sibling > to that given namespaces head device. > Failover is triggered either due to keep-alive timer expiration or RDMA > timeout. Selection is of device is based on first available standby > device. As of present this implementation support Active-Standby multi- > path model. > On selection of device, a command is sent to target for acknowledgement > of active device. In meanwhile if any IO are received, they get queued > under head multipath device congestion queue and processed as soon as > a single path becomes active. In scenario where new active path results > in failure, it will cancel those IOs after multiple retries. > > I have gone through the solutiion Christoph Hellwig suggested and this > one follow similar path except for it doesn't require any change from > kernel and will work prior version of kernel. > It can made standalone module and be used with other block devices as > we open any block device handle. > > It has been tested with interface up/down on host and target, target > node crash and disconnect command. Performance has been tested with > multiple multipath devices on single host and there is un-noticeable > difference in numbers. > In general I am _not_ in favour of this approach. This is essentially the same level of multipath support we had in the old qlogic and lpfc drivers in 2.4/2.6 series, and it took us _years_ to get rid of this. Main objection here is that it will be really hard (if not impossible) to use the same approach for other subsystems (eg SCSI), so we'll end up having different multipath implementations depending on which subsystem is being used. Which really is a maintenance nightmare. I'm not averse to having other multipath implementations in the kernel, but it really should be abstracted so that other subsystems can _try_ to leverage it. Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg)
[PATCH 00/10] nvme multipath support on top of nvme-4.13 branch
Hello everyone, Please find patchset for supporting Multipathing using nvme namespace and nvme controller. Basic idea behind the design and implementation is to re-use the same logic which is between generic nvme namespace and it's controller. One controller to many namespaces. Similarly, this implementation uses one multipath controller and sibling namespaces. It creates a head multipath nvme namespace as soon as it find a nvme namespace during enumeration which has property of being shared. Prior to creating new head multipath device, nvme namespace check for match- ing NGUID, if it finds one, this nvme namespace is added as a sibling to that given namespaces head device. Failover is triggered either due to keep-alive timer expiration or RDMA timeout. Selection is of device is based on first available standby device. As of present this implementation support Active-Standby multi- path model. On selection of device, a command is sent to target for acknowledgement of active device. In meanwhile if any IO are received, they get queued under head multipath device congestion queue and processed as soon as a single path becomes active. In scenario where new active path results in failure, it will cancel those IOs after multiple retries. I have gone through the solutiion Christoph Hellwig suggested and this one follow similar path except for it doesn't require any change from kernel and will work prior version of kernel. It can made standalone module and be used with other block devices as we open any block device handle. It has been tested with interface up/down on host and target, target node crash and disconnect command. Performance has been tested with multiple multipath devices on single host and there is un-noticeable difference in numbers. The git tree for above changes is available at https://github.com/paviliondata/nvme.git Branch nvme-4.13 Web interface https://github.com/paviliondata/nvme/tree/nvme-4.13 Please peruse through the patchset and comment. Anish M Jhaveri (10): Initial multipath implementation. RDMA timeout triggers failover. Failover timeout and time between failover same ns. NVME command to set namespace active. Init multipath head controller. Init multipath head namespace. Selection of active namespace. Multipath make request, retry and cancel IO. Sysfs entries for multipath head Reserved tag for active ns command drivers/nvme/host/core.c | 1245 +- drivers/nvme/host/nvme.h | 37 +- drivers/nvme/host/rdma.c |5 +- 3 files changed, 1259 insertions(+), 28 deletions(-) -- 2.9.3