On 06/11/2013 12:43 PM, Eddie Wai wrote: > > On Tue, 2013-06-11 at 12:25 -0500, Mike Christie wrote: >> On 6/10/13 10:47 PM, Vikas Chaudhary wrote: >>> >>> >>> -----Original Message----- >>> From: Eddie Wai <w...@broadcom.com> >>> Date: Wednesday 5 June 2013 7:00 AM >>> To: "shyam_i...@dell.com" <shyam_i...@dell.com> >>> Cc: "open-iscsi@googlegroups.com" <open-iscsi@googlegroups.com>, Vikas >>> <vikas.chaudh...@qlogic.com>, Lalit Chandivade >>> <lalit.chandiv...@qlogic.com>, Ravi Anand <ravi.an...@qlogic.com>, >>> Poornima Vonti <poornima.vo...@qlogic.com>, Manish Rangankar >>> <manish.rangan...@qlogic.com>, Adheer Chandravanshi >>> <adheer.chandravan...@qlogic.com> >>> Subject: RE: [RFC_V5 PATCH 1/3] scsi_transport_iscsi: Add flash node mgmt >>> support >>> >>>> >>>> On Wed, 2013-05-29 at 14:37 -0700, shyam_i...@dell.com wrote: >>>>> >>>>>> -----Original Message----- >>>>>> From: open-iscsi@googlegroups.com [mailto:open-iscsi@googlegroups.com] >>>>>> On Behalf Of Mike Christie >>>>>> Sent: Wednesday, May 29, 2013 1:30 PM >>>>>> To: open-iscsi@googlegroups.com >>>>>> Cc: Eddie Wai; vikas.chaudh...@qlogic.com; >>>>> lalit.chandiv...@qlogic.com; >>>>>> ravi.an...@qlogic.com; poornima.vo...@qlogic.com; >>>>>> manish.rangan...@qlogic.com; Adheer Chandravanshi >>>>>> Subject: Re: [RFC_V5 PATCH 1/3] scsi_transport_iscsi: Add flash node >>>>> mgmt >>>>>> support >>>>>> >>>>>> On 05/29/2013 12:23 PM, Eddie Wai wrote: >>>>>>> >>>>>>> On Wed, 2013-05-29 at 10:26 -0500, Mike Christie wrote: >>>>>>>> On 05/28/2013 07:35 PM, Eddie Wai wrote: >>>>>>>>> Hello Mike, Vikas, and all, >>>>>>>>> >>>>>>>>> Thanks for the great work in creating the flash node mechanism! >>>>>>>>> >>>>>>>>> To extend the conversation we had to add support for software and >>>>>>>>> other offload solutions that requires iscsid/iscsiadm to create >>>>> the >>>>>>>>> sessions, the following needs to be further discussed: >>>>>>>>> >>>>>>>>> 1. Flash node creation. >>>>>>>>> >>>>>>>>> The current solution relies on the transport driver to initiate >>>>> the >>>>>>>>> flash node sysfs creation upon iscsi_host allocation. This >>>>> presents >>>>>>>>> a fundamental problem for software iSCSI as new iscsi_host >>>>> instance >>>>>>>>> won't get created until there's a session initiation. >>>>>>>>> >>>>>>>>> Per our conversation, I think it makes sense to move the flash >>>>> node >>>>>>>>> initiation code altogether to a separate module like how its done >>>>>>>>> for ibft. The new module shall cycle through each existing iSCSI >>>>>>>>> host and query the corresponding transport to fill the flash node >>>>>>>>> sysfs entries specific to that host. Perhaps via a new transport >>>>> callback >>>>>> or so. >>>>>>>> >>>>>>>> Agree. >>>>>>>> >>>>>>>>> >>>>>>>>> Since there won't be any pre-existing host created for software >>>>>>>>> iSCSI, this needs to be handled differently. Instead, the new >>>>>>>>> module will also need to cycle through each network interfaces >>>>>>>>> present and query for the flash node content separately. >>>>>>>>> >>>>>>>>> To accomplish this, each network interface will need to be made >>>>>>>>> aware of flash nodes and also provide a way for the new module to >>>>>>>>> read out the flash node content. Per our conversation, this can >>>>> be >>>>>>>>> done via an extension of the ethtool utility. For unsupported >>>>>>>>> network drivers, this >>>>>>>> >>>>>>>> I do not remember that discussion. Has it been discussed with the >>>>>>>> netdev people already? >>>>>>> This has only been discussed internally so far, but we believe >>>>> adding >>>>>>> a new ethtool extension for this flash memory access is one logical >>>>>>> way that the netdev community can accept. >>>>>>>> >>>>>>>> Why does the new module need to cycle through each net device? >>>>> Can't >>>>>>>> a net driver that knows it supports this just call into the new >>>>>>>> module to register itself when it is loaded? When it registers it >>>>> can >>>>>>>> create any sysfs/netlink stuff needed so userspace can detect it >>>>> and >>>>>> access it. >>>>>>> That would work too, but our proposal essentially is tailored toward >>>>>>> minimizing any storage specific code in the network drivers. >>>>>>> >>>>>>> Note that our proposal is to add an ethtool extension which will >>>>>>> provide read/write access directly to the flash memory. It will not >>>>>>> do any sysfs creation or parameter qualification. It only provides >>>>> a >>>>>>> gateway to the flash memory, that's it. It will be up to the new >>>>>>> module to initiate, create, and manage over those sysfs parameters. >>>>>> >>>>>> Sounds ok to me. >>>>>> >>>>>>> >>>>>>> Perhaps we can add some minor initiation code in the network driver >>>>> to >>>>>>> perhaps 'register' some flag so the new module will only have to >>>>> cycle >>>>>>> through a list of supported network interfaces only. >>>>>> >>>>>> It is ok. I was just thinking along the lines of either ethtool or >>>>> iscsi mod only. I >>>>>> cannot think of a major issue to probe with ethtool from userspace >>>>> like you >>>>>> suggested before. The only issue might be if we have to do some sort >>>>> of >>>>>> probing and checking if this is supported takes a while (like if we >>>>> have to do >>>>>> some fw command that takes several to 10-15 seconds each time). For >>>>> that >>>>>> case we do not want to have to probe every device during boot or the >>>>>> boot/distro people will not be happy. >>>>>> >>>>> >>>>> Second that.. >>>>> >>>>> One alternative is that the network driver registers to the new module. >>>>> I would prefer that the new module is loaded already so it can >>>>> enumerate the /sysfs entries correctly. >>>>> >>>> Although we have not proposed this via the netdev ML yet, however, the >>>> current suggestion to only add ethtool extension to support this is >>>> gaining traction. >>>> >>>> As far as latency goes, a simple for_each_netdev loop to probe each >>>> netdev's ethtool_ops for these new extensions (like get/set_flash_node) >>>> shouldn't incur too much cost for unsupported nics. However, for the >>>> case when there are many supported nics in the system, we would then >>>> have to decide how to best handle the delay. We might have to rely on >>>> defer tactics or impose a hard limit on the number of flash targets to >>>> support on a system. This should be easy to govern via the bios. >>>> >>>> How about for HBAs? We probably want to employ the same top-down >>>> approach and have the new module cycle through each created host and >>>> pull for the flash node targets. The alternative is to call into the >>>> new module to create those flash node sysfs entries upon iscsi_host >>>> allocation. What would be the best way to do this? >>> >>> >>> I think iSCSI HBA Driver call into the new module to create flash node >>> sysfs entries upon iscsi_host allocation is better way to do it. >>> >>> >> >> I agree. >> > With HBAs, the bottom-up approach seems to be more logical and probably > would incur lesser of an impact for all vendors. But for non-HBAs, it > makes better sense for the new module to query through the netdevs > instead since the networking drivers are passive in this sense. > > So, it looks like the best approach would be to fundamentally handle > them differently. >
One related question. Are we also going to have the iscsi hbas's implement some ethtool ops for this, or will only drivers that interact with a netdev do that? If the iscsi hbas do it will it use a dummy netdev or are we going to modify the ethtool ops to not use a netdev. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
