Re: nvmf question - synchronization between target/initiator regarding partitions
On 08/10/2017 06:16 AM, Christoph Hellwig wrote: > On Mon, Aug 07, 2017 at 02:29:47PM -0300, Guilherme G. Piccoli wrote: >> Thanks for your feedback Hannes, agreed! > > And btw, you'll see similar results with the SCSI target or nbd, > so it's not really nvme specific. Thanks, I agree - noticed the same stuff. I've used nvmf as a "trigger" to the subject, in order to discuss a possible generic solution. But since everything is working as expected, no need to pursue a "fix" heheh
Re: nvmf question - synchronization between target/initiator regarding partitions
On Mon, Aug 07, 2017 at 02:29:47PM -0300, Guilherme G. Piccoli wrote: > Thanks for your feedback Hannes, agreed! And btw, you'll see similar results with the SCSI target or nbd, so it's not really nvme specific.
Re: nvmf question - synchronization between target/initiator regarding partitions
Thanks for your feedback Hannes, agreed! Cheers, Guilherme
Re: nvmf question - synchronization between target/initiator regarding partitions
On 08/07/2017 03:42 PM, Guilherme G. Piccoli wrote: > We observed that it's possible to perform partition operations in both > nvmf target and initiator block devices, like creating and deleting > partitions. > > But there is no sync mechanism between target and initiator regarding > the partitions operations. After creating a partition on initiator, for > example, we don't see it on target side. We could format it like ext4 on > initiator, and still target cannot see it. It's possible to trigger a > BLKRRPART ioctl on target, which end up calling revalidate_disk() on > nvme driver and then partitions are perceived. > > So, question: is this behavior expected/acceptable? Is it completely up > to userspace to deal with the synchronization between host/target? > I think answer might be yes since partitions are a higher level of > abstraction than nvmf (which is purely block device aware). > Yes, this is to be expected. After all, one could argue that no partition information should ever be visible on the target seeing that the device is exported. As the initiator has exclusive access, the target has no business looking at the contents; in fact, this might lead to unexpected corruptions if things like udev jump in on the target side and try to 'correct' things with journal replay etc. > But if kernel could/should jump in, we possibly could try to issue a > revalidate_disk on target whenever this operation is performed on > initiator side (and vice-versa). I confess I couldn't find such sync > idea in NVMe over fabrics spec, though. > Also, reading NVMe spec 1.3, we do have the optional feature > "reservations", but seems it doesn't mention target (only multiple hosts). > See above. Never try this :-) Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg)
nvmf question - synchronization between target/initiator regarding partitions
We observed that it's possible to perform partition operations in both nvmf target and initiator block devices, like creating and deleting partitions. But there is no sync mechanism between target and initiator regarding the partitions operations. After creating a partition on initiator, for example, we don't see it on target side. We could format it like ext4 on initiator, and still target cannot see it. It's possible to trigger a BLKRRPART ioctl on target, which end up calling revalidate_disk() on nvme driver and then partitions are perceived. So, question: is this behavior expected/acceptable? Is it completely up to userspace to deal with the synchronization between host/target? I think answer might be yes since partitions are a higher level of abstraction than nvmf (which is purely block device aware). But if kernel could/should jump in, we possibly could try to issue a revalidate_disk on target whenever this operation is performed on initiator side (and vice-versa). I confess I couldn't find such sync idea in NVMe over fabrics spec, though. Also, reading NVMe spec 1.3, we do have the optional feature "reservations", but seems it doesn't mention target (only multiple hosts). Thanks in advance for any ideas on this subject. Cheers, Guilherme