Re: [PATCH] minimal SAS transport class
On Fri, Aug 26, 2005 at 03:24:06PM -0400, Jeff Garzik wrote: Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. What's a scsi_domain_device? Anyway, we don't need any HCIL tuple for all of the above. SG_IO is done on a blockdevice node, /dev/sg is done on a chardevice node. Each of them are attached to a request_queue that's known at the time their -probe routine is setup - no need for HCIL here _at all_. There's actually only few userland interfaces that use HCIL addressing: the scanning through /sys/.../scan (or the old /proc/scsi/scsi variant), using WWNs for FC and SAS here would be much nicer, but there's a huge backwards-compatiblity issue - we'll probably have to support the old variant forever. Besides that there's probably only SCSI_IOCTL_GET_IDLUN, which is superceeded by sysfs but probably still has tons of users in hardware probing, managment utilities and all kinds of userland. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/29/05 13:11, Christoph Hellwig wrote: On Fri, Aug 26, 2005 at 03:24:06PM -0400, Jeff Garzik wrote: Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. What's a scsi_domain_device? Anyway, we don't need any HCIL tuple for all of the above. SG_IO is done on a blockdevice node, /dev/sg is done on a chardevice node. Each of them are attached to a request_queue that's known at the time their -probe routine is setup - no need for HCIL here _at all_. There's actually only few userland interfaces that use HCIL addressing: the scanning through /sys/.../scan (or the old /proc/scsi/scsi variant), using WWNs for FC and SAS here would be much nicer, but there's WWNs *must not* be visible _at_all_ to anyone above the SAS layer, in the kernel. That is, the transport addressing method *must not* be visible to SCSI Core or anyone above it, in the kernel. Else you'll end up with another HCIL... (if you get the irony). a huge backwards-compatiblity issue - we'll probably have to support the old variant forever. Besides that there's probably only SCSI_IOCTL_GET_IDLUN, If there's no userspace dependency, the kernel can do anything it wants. Since HCIL is SCSI Core crud, let SCSI Core invent HCIL tuples and support them. Luben which is superceeded by sysfs but probably still has tons of users in hardware probing, managment utilities and all kinds of userland. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Mon, Aug 29, 2005 at 01:20:50PM -0400, Luben Tuikov wrote: What's a scsi_domain_device? Anyway, we don't need any HCIL tuple for all of the above. SG_IO is done on a blockdevice node, /dev/sg is done on a chardevice node. Each of them are attached to a request_queue that's known at the time their -probe routine is setup - no need for HCIL here _at all_. There's actually only few userland interfaces that use HCIL addressing: the scanning through /sys/.../scan (or the old /proc/scsi/scsi variant), using WWNs for FC and SAS here would be much nicer, but there's WWNs *must not* be visible _at_all_ to anyone above the SAS layer, in the kernel. That is, the transport addressing method *must not* be visible to SCSI Core or anyone above it, in the kernel. Please read the sentence again. I said accept them in the scan attribute, that does _NOT_ mean adding knowledge to the scsi layer. In fact if you through the linux-scsi archives about the problems those attributes had with the FC transport class I suggested moving the parsing of these attributes into the transport class, which would be the preferred method to implement this. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/29/05 13:16, Christoph Hellwig wrote: No need to do silly renaming, but yes, moving creating of scsi_target I'd do the silly renaming. I'd also create struct scsi_domain_device, and do scsi_register_domain_device(). You know, clean slate... structures to the transport does make sense. It's kinda implicit There is *nothing* implicit in a Software Project Specification! Everything must be completely explicit. in the todo list I posted. I also don't really see the point of the embedded kobject. You will, once I post my code. Think, - Hotplugging. - More than one owner from above. That is, on a transport you can have a diverse set of devices. What if queuecommand() was *not* the only way to send a task to the device? ;-) We actually already have a list in the scsi_target that chains of the scsi_device, .devices in scsi_target and .same_target_siblings in scsi_device. We just need to use it everywhere. Yes, I've seen all this. And I'm sorry to say, but it is *UGLY AS HELL!* As I said: before implementing an object by a structure in a software project one has to ask themselves what that object is? How will it play with the rest of the objects existsing or to be designed? What are the dependencies and what is the dependency graph? Etc. First it starts with a white sheet of paper, pencil on one side and a spec on the other. Yes. that's what I ment with my item (3) (sorry, I hate all this techno-babble, simple language is much easier to understand normally) Ok, sorry, it's that software specificaion tehcno-babble thing talking through me again. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Mon, 2005-08-29 at 18:16 +0100, Christoph Hellwig wrote: No need to do silly renaming, but yes, moving creating of scsi_target structures to the transport does make sense. It's kinda implicit in the todo list I posted. I also don't really see the point of the embedded kobject. You might want to dust off the Mike Christie patch that did this. The one that eventually went in the tree was James Smarts to stop FC drivers oopsing on remove/add lun. There already is an embedded kobject in the scsi_target structure (it's in struct device) and that's what we use to make it visible in the sysfs tree. James - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/29/05 13:31, Luben Tuikov wrote: On 08/29/05 13:16, Christoph Hellwig wrote: No need to do silly renaming, but yes, moving creating of scsi_target I'd do the silly renaming. I'd also create struct scsi_domain_device, and do scsi_register_domain_device(). You know, clean slate... structures to the transport does make sense. It's kinda implicit There is *nothing* implicit in a Software Project Specification! Everything must be completely explicit. in the todo list I posted. I also don't really see the point of the embedded kobject. You will, once I post my code. Think, - Hotplugging. - More than one owner from above. That is, on a transport you can have a diverse set of devices. What if queuecommand() was *not* the only way to send a task to the device? ;-) Forgot to mention one more thing, which I'm sure you're aware of: *If* the kobject hierarchy is set right, then kobject_get() gets this object and _all_ objects which are parents of this object. And kobject_put() puts all objects which are parent of this object, including calling the release method of each.(*) You'll need this to support hotplugging on the fly... Luben (*) Thus if your sysfs tree is built as the physical world looks(**), you lock the object(s) when you use them, so that if any intervening object is removed and you get an event notification for it, you know what to do... ;-) (**) Which the transport class doesn't give you, since it was never _designed_ for that purpose... unless of course you slice-it-and-dice-it pretty well. ;-) We actually already have a list in the scsi_target that chains of the scsi_device, .devices in scsi_target and .same_target_siblings in scsi_device. We just need to use it everywhere. Yes, I've seen all this. And I'm sorry to say, but it is *UGLY AS HELL!* As I said: before implementing an object by a structure in a software project one has to ask themselves what that object is? How will it play with the rest of the objects existsing or to be designed? What are the dependencies and what is the dependency graph? Etc. First it starts with a white sheet of paper, pencil on one side and a spec on the other. Yes. that's what I ment with my item (3) (sorry, I hate all this techno-babble, simple language is much easier to understand normally) Ok, sorry, it's that software specificaion tehcno-babble thing talking through me again. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/26/05 21:53, Jeff Garzik wrote: Luben Tuikov wrote: On 08/26/05 15:24, Jeff Garzik wrote: Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These The request queue is associated with the LU, not the scsi_domain_device. When SCSI Core discovers the LU, it sets up the request queue for it, etc. Again this is the role of SCSI Core, not messing up with transport specific stuff. upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. I don't get this. More basically... An in-kernel C pointer, to a SCSI target device, is not sufficient in all cases to address a target. This plays out most often in userland interfaces such as ioctls. 1. Why do I care about this stuff, when I'm so low in the layering infra? 2. I thought ioctls are bad. 3. So you're saying that there's an ioctl which addresses a SCSI target device by HCIL? Which one is it please? That is the heart of the routing/addressing that the SCSI core must perform. Disagree: now: scsi_device -- request_queue, then: struct LU -- request_queue. The LU points to the domain_device (as its parent). The domain_device has a void *lldd_dev in it. The current SCSI code largely already has this stuff. No, it has no concept of those things. I mean, look at how scsi_target is treated and implemented... Before one start writing code about something (scsi_target) one should ask themselves what is that something (scsi_target)?, and how does it play with the rest of the objects I want to represent?. You can search the archives of linux-scsi and you'll see how many times I've asked about true scsi_target representation, and how many times I've been rejected by the SCSI maintainers. Now the need for such an object is even more dire, when you have _one_more_ SCSI transport. It started with iSCSI... No specs, just a comment from IRC. Oh, I see... this was one of those IRC sessions you had with James B, which you talked about before. I'd suggest sitting down with a fresh copy of SAM and SPC, reading them 20 times, then looking at this picture: http://www.t10.org/scsi-3.gif and then seeing how _a_ SCSI Core should behave. (host,string) could succeed in transporting both HCIL and non-HCIL target identifiers. Broken. BTW, none of what I'm saying here has changed for the last 5 years. It's all the same old stuff. Now its SAS, back then it was iSCSI. What the maintainers here fail to see is that it is all SAM and they fail to see how it all plays together with the transports and then with the interconnects. The reason we don't see eye to eye is that we don't come off the same base. Some people here re-read every revision of SAM, SPC, etc, when it comes out, others do not. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/26/05 21:39, Jeff Garzik wrote:
Luben Tuikov wrote:
On 08/26/05 14:48, Jeff Garzik wrote:
No host numbers, no routing information. This is all
transparent to SCSI Core, and NONE of its business.
Routing is an essential part of the SCSI core's duties.
[I'm not a big fan of reading mixed-message emails, but what can you do...]
The SCSI core is the resource manager responsible for routing messages
[CDBs] to/from LLDs based on scsi-specific device address. This
includes resolution of kernel-specific identifiers (device major/minor,
etc.) into s.s.d.a.. This also includes direct use of
This particular is the task of sd.c. How it does it is
sd.c. job. Not SCSI Core.
No. sd, sr, st, and sg all use the -common- infrastructure to execute
tasks and return results. That common infrastructure is part of the
SCSI core.
The SCSI layer itself is a marraige between
device classes -- sd, sr, st, sg
transport classes -- common per-transport code
drivers -- executes tasks via transport class
glue -- the myriad functions that tie the above 3 together
Yes, this is the current infrastructure and is quite messy without
clear separation between layers.
SCSI layer should sit above any transport specific layer and should
have no knowlege about the transport specifics. Read SAM.
All transport-specific knowledge that is common across hardware vendors
should be in the transport class.
First, there is no such thing as transport specific knowlege common
accross vendors -- maybe you mean same-transport, different vendor?
Yes, this is what I'm driving at: same-transport, same transport-layer,
different vendors, *but* clear separation between them:
FS/user Block/char Command set drivers SCSI Core transport layer
LLDD/transport interconnect physical world.
Transport layer is *not* James B's transport class, because it, transport
class, falls short of representing the specific concepts that a transport
may have. Case in point: SAS.
I'd suggest looking at those figures:
SAM4r02, page 2, Figure 1 and Figure 2.
SPC4r00, page 1, Figure 1.
SAS1r09, page 33, Figure 10 -- yellow highlight is what should be SCSI Core.
The SCSI core uses the transport
class to perform transport-specific actions.
And it should absolutely not. Because the layering infratructure would be
broken
then. Or would SCSI Core know about every transport there is.
Transport specific actions should be performed only by the transport
layer, which sits _below_ SCSI Core.
The SCSI core is the common point for exporting bus topology via
transport classes.
Which is again: wrong.
A transport implementation sits below SCSI Core and exports
topology info in a unified way for all vendors implementing
a transport.
Moving away from HCIL requires a lot of thought, including thinking
about userland app breakage -- a big deal in Linux.
I never contended that userspace should be moved away from HCIL.
Then, by implication, SAS and FC must continue to maintain HCIL-device
maps.
And I repeat: those should be done by SCSI Core, simply because:
- They (HCIL) are _invented_ by SAS and FC and USB and Firewire, etc.
- They (HCIL) are crud which only SCSI Core requires.
SAM is already mostly there. -queuecommand is already a pretty good
execute_task().
Jeff *do not spread FUD*! SCSI Core doesn't know about SAM to save its
life!
The situation hasn't changed one bit for the last 5 years!
There are no TMF implementions, the layering infrastructure is wrong, etc, etc.
Most easily this would be done by implementing a bunch of
new-way-to-do-it functions. The request_queue wouldn't care,
and old LLDD can use the old interface, and new ones can use
the new interface.
Disagree. Just follow the TODO list Christoph outlined, plus figure out
Christoph was asking me if that list was ok -- just to be clear.
how to handle SG_IO and /dev/sg sanely.
We don't need yet more
if (new way) {
...
} else {
...
}
code blocks :)
Hmm, no. There will never be one such block. The new way and
the old way will be completely unaware of each other. Once a
transport layer starts registering SCSI domain devices with the
new SCSI Core, it will just go from there.
_Plus_ it will be so much _shorter_ and straightforward.
HCIL addressing gunk largely belongs in SPI transport class, along with
scsi_scan_host()
SPI transport _layer_ will handle finding devices on the SPI domain.
It then registers those with SCSI Core *in no different way* than
SAS or USB or FC or Firewire would register them. Then SCSI Core
sticks to SPC and does LU discovery, request_queue set up (or whatever),
and announces the LUs to the Command set drivers. (Look at the figures
I mentioned above.)
[each transport class should build its own topology].
Agree.
This achieves the result you want to achive: new-way-to-do-it functions
live in
Re: [PATCH] minimal SAS transport class
Jeff Garzik wrote: (host,string) could succeed in transporting both HCIL and non-HCIL target identifiers. What is the meaning of host there? -- Stefan Richter -=-=-=-= =--- ===-= http://arcgraph.de/sr/ - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Jeff Garzik wrote: Luben Tuikov wrote: I never contended that userspace should be moved away from HCIL. Then, by implication, SAS and FC must continue to maintain HCIL-device maps. Yes, but not in the SAS/ FC/ iSCSI/ USB/ SBP-2/... low-level. HCIL does not exist there. HCIL is only a bonus for the interface to userspace. Therefore HCIL should be sticked to those devices by the core, somewhere close to the surface to userspace. The low-level should never care that arbitrary HCIL tuples should be added for some applications in userspace. Moreover... What I contend is that _internally_ SCSI Core start moving away from HCIL and towards SAM. ...the whole notion that low level provides hosts, channels etc. is often wrong. usb_storage and sbp2 always had to resort to a one target = one host implementation for several reasons. But there is no host, there is only a bunch of targets. [...] You will never be able to eliminate transport-specific code. That's the whole point of transport classes: encapsulate common transport code. Call it a transport library, rather than a class, if class gives people the shivers. Example: I have access to SAS+SATA host controllers from Adaptec and Company X. Both are largely software-based, directly controlling the PHY ports and manually performing all SAS+SATA discovery. Yes. And there would be transport classes which do not control hardware at all but rather push a high-level protocol (iSCSI, USB mass storage, SBP-2...) over another whole infrastructure which they share with many unrelated protocols. These SCSI transports are separated from hardware by several more layers of abstraction, at least two of those layers being in-kernel programming interfaces. -- Stefan Richter -=-=-=-= =--- ==-== http://arcgraph.de/sr/ - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Jeff Garzik wrote: An in-kernel C pointer, to a SCSI target device, is not sufficient in all cases to address a target. This plays out most often in userland interfaces such as ioctls. The C pointer is all the interface between core and low-level needs. All other means of addressing (IDs, labels, ... of which HCIL is only one kind) are only meaningful to userspace. The labels may be _generated_ at any level (low, core, high) depending on their context, but they are _used_ only in userspace-kernel interfaces. The core provides the map of devices-labels. -- Stefan Richter -=-=-=-= =--- ==-== http://arcgraph.de/sr/ - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/24/05 05:12, Christoph Hellwig wrote: On Tue, Aug 23, 2005 at 08:02:39PM -0400, Luben Tuikov wrote: In effect FC tells SCSI Core: here is a device I do not know anything about, other than the fact that it is on the fabric and it has a SCSI Target (or Initiator) Port: how you get to is is _my_ business, what is done with or to the device is _your_ business. Then SCSI Core does LU discovery using SPC and SAM on the device using Execute Task. All this _completely_ eliminates the need for HCIL. How the FC layer identifies the device is the FC's layer problem. How SCSI Core identifies the device is SCSI Core's problem. HCIL should have never been available to SCSI Core in the first place, but then again SAM didn't exist back then. Ok, Luben, here's the TODO list to get there: (1) add a sdev_printk that takes a struct scsi_device * and prints a sensible name for it. Later we might or might not make this a transport hook, but for now it gets rid of all these -id references in LLDDs (2) add a variant of scsi_scan_target that takes a scsi_target * to scan, make the old scsi_scan_target a wrapper around it and introduce similar wrappers for transport-specific target identifiers (e.g. WWNs for FC and SAS) Hi Christoph, Yeah this sounds cool. Do you think it would be beneficial to revisit struct scsi_target to something like struct scsi_domain_device and have a kobject in it. Then the FC/SAS/iSCSI/USB layer upon finding a domain device, they call a suitable function which allocates them struct scsi_domain_device, which sets the release method for the kobject embedded. Then they call SCSI Core to register the domain device. (All in all, the transport layer may also do kobject_get.) SCSI Core does kobject_register (thus kobject_get) and starts doing LU discovery for the device (all through the pointer to struct scsi_domain_device), and thus the LUs are discovered and struct scsi_device is created for each one. Then if the device is unplugged from the domain, the protocol layer does whatever is necessary, including kobject_put(), and then calls the unregister function in SCSI Core to unregister the device. SCSI Core does whatever it wants and then does kobject_unregister(kobject_put), and the device is gone. If OTOH, the unplug event came from userspace, SCSI Core does whatever is necessary to finish with the device, and then calls the slave_destroy() function (you get the idea), and then does kobject_put. Of course the order of those doesn't have to be quite like so. I've all this implemented for the _current_ architecture of SCSI Core. The aforementioned was a draft to extend (move) domain devices _and_ LUs into SCSI Core where they belong, as opposed to the transport layer doing LU discovery _as_well_as_ domain device discovery and giving only LUs to SCSI Core. (3) fix a bunch of internal functions that use -id despite not actually needing it, e.g. starget_for_each_device Yeah, that sounds cool. In effect, all you'll have is a struct list_head of LUs, and that list_head would live in the struct scsi_domain_device. (4) eventually move the traditional scsi_scan_target and other functions that lookup a scsi_target based on -id into the spi transport class Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. The current slave_alloc/slave_configure/slave_destroy implementation is _very_good_, but slight adjustments would have to be made when the domain device concept arrives at SCSI Core -- nothing unprecedented. If anything, it would be a _simplification_, and I'm actually in _favour_ of keeping the current infra in place as well -- no big, sudden changes. (5) add an SPI-specific taget ID accessor and move all drivers over to it Yeah, sure. The transport layer would actually assign these ids as part of it doing domain discovery. They would be some opaque string in some opaque format, WWN, H:C:I:L, X,Y, etc, which the transport layer tacked to the domain device before announcing that it exists to SCSI Core. SCSI Core can feel free to add more and other various labels if it wishes to do so. Imagine a struct list_head of labels. Luben P.S. Transport class != Transport layer. SCSI Core transport layer (FC/SAS/USB/iSCSI) LLDD/transport interconnect physical world. Anything to the left is completely unaware of internals of anything to the right. Access to the right is provided as dictated by some spec. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/25/05 16:06, Jeff Garzik wrote: Stefan Richter wrote: Luben Tuikov wanted... to move _away_ from HCIL. So that addressing of devices in SCSI Core is _completely_ independent of addressing at the protocol layer. I second that. A move away from forced HCIL addressing would be a good thing. However, its impossible to completely move away from addressing, as userspace and the SCSI core need ways to route CDBs to devices based on address. They can use _anyone_ label in the label list of the LU. The label itself is opaque to SCSI Core. It may mean something to users, it may mean something to the protocol layer, but to SCSI Core it is completely opaque, and SCSI Core should _not_ try to interpret it/them. Anyone, at any layer can add a label: protocol, SCSI Core, Command set drivers, block subssytem, FS. In fact, there is a dire need for this from several storage vendors who want customers to look at the face of the storage box (physical world) and then look into the OS/Linux and see/know the device (correspondence). Doing the HCIL juggle is the _ugliest_ thing currently amongst all OSes. Sadly, it had to take 5 years... Luben P.S. Everyone, please don't remove CC entries from the email header, else they go into my linux-scsi folder instead of my Inbox and it is hard to track threads I'm participating in. If you didn't add it, don't remove it. :-) Thanks! - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Fri, 2005-08-26 at 12:43 -0400, Luben Tuikov wrote: A move away from forced HCIL addressing would be a good thing. However, its impossible to completely move away from addressing, as userspace and the SCSI core need ways to route CDBs to devices based on address. They can use _anyone_ label in the label list of the LU. I think what Jeff means is that the mid-layer needs to know which LLD to send the CDB to. This is the routing information (and is really just the host number). The LLD takes the rest of the information to work out where it's sending the CDB. For SPI this is often just PUN and LUN. For FC it's usually WWPN and LUN. Currently the information we send down is the endpoint, represented by the scsi_device. The label itself is opaque to SCSI Core. It may mean something to users, it may mean something to the protocol layer, but to SCSI Core it is completely opaque, and SCSI Core should _not_ try to interpret it/them. Labelling devices is policy. That's the bit we're trying to push up to the user via tools like udev. The routing information we need is already inherent in the structure of the scsi generic device tree. So, by and large, the mid layer doesn't care what the numbers contained within the struct scsi_device are. It just uses a scsi_device to send a CDB. The LLDs however, do care. SPI LLDs want to find channel, id (target) and lun numbers. FC drivers map id to wwpn internally and so would probably quite like a way of sticking the wwpn into the id instead of having a fictitious number. The point is that labels are user policy and managed at user level. The representations used in the generic device tree shows what the kernel actually uses and should be the preferred form for the transport. So what's really needed is a scheme that allows the LLDs to use the id transparently without having to map it to what they really want while not breaking all the current SPI users (or which patches up every SPI driver). James - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote: On 08/26/05 13:22, James Bottomley wrote: On Fri, 2005-08-26 at 12:43 -0400, Luben Tuikov wrote: A move away from forced HCIL addressing would be a good thing. However, its impossible to completely move away from addressing, as userspace and the SCSI core need ways to route CDBs to devices based on address. They can use _anyone_ label in the label list of the LU. I think what Jeff means is that the mid-layer needs to know which LLD to send the CDB to. No, I thought he meant about user space apps, *NOT* SCSI Core. Since, the transport found the device on the domain (NOT LU!) it then calls SCSI Core to register it. So you have: task-scsi_domain_device-lldd-lldd_execute_task(task). This is the routing information (and is really just the host number). No host numbers, no routing information. This is all transparent to SCSI Core, and NONE of its business. Routing is an essential part of the SCSI core's duties. The SCSI core is the resource manager responsible for routing messages [CDBs] to/from LLDs based on scsi-specific device address. This includes resolution of kernel-specific identifiers (device major/minor, etc.) into s.s.d.a.. This also includes direct use of userspace-provided identifiers as s.s.d.a., such as via SG_IO ioctl. Moving away from HCIL requires a lot of thought, including thinking about userland app breakage -- a big deal in Linux. Ask yourself where all these HCIL-addressed CDBs come from... each one of those CDB submittors must be updated from HCIL addressing/routing to transport-specific. Jeff - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. That is the heart of the routing/addressing that the SCSI core must perform. Right now the addressing is hardcoded to HCIL. But that can be changed... One proposal was to use (host,string) identifiers. Jeff - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/26/05 15:24, Jeff Garzik wrote: Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These The request queue is associated with the LU, not the scsi_domain_device. When SCSI Core discovers the LU, it sets up the request queue for it, etc. Again this is the role of SCSI Core, not messing up with transport specific stuff. upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. I don't get this. That is the heart of the routing/addressing that the SCSI core must perform. Disagree: now: scsi_device -- request_queue, then: struct LU -- request_queue. The LU points to the domain_device (as its parent). The domain_device has a void *lldd_dev in it. Right now the addressing is hardcoded to HCIL. But that can be changed... One proposal was to use (host,string) identifiers. Who? Who is proposing this? I never saw an email to SCSI Core about this proposal? Is there any more information about this proposal and what is the justification of it? Any specs and docs? Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/26/05 14:48, Jeff Garzik wrote: No host numbers, no routing information. This is all transparent to SCSI Core, and NONE of its business. Routing is an essential part of the SCSI core's duties. [I'm not a big fan of reading mixed-message emails, but what can you do...] The SCSI core is the resource manager responsible for routing messages [CDBs] to/from LLDs based on scsi-specific device address. This includes resolution of kernel-specific identifiers (device major/minor, etc.) into s.s.d.a.. This also includes direct use of This particular is the task of sd.c. How it does it is sd.c. job. Not SCSI Core. userspace-provided identifiers as s.s.d.a., such as via SG_IO ioctl. Ah, yes, I see. So the question is, how do we fit SG_IO ioctl? SG_IO can be transport/protocol agnostic _if_ SCSI Core gets the architecture right. I.e. if I show you a picture of the objects out there in the SCSI domain, you can just point to one and send something to it. That picture will be painted by the transport layer. SCSI Core is _completely_ unaware of all this! This is how you can accomodate SMP and any future protocols that can come your way. Moving away from HCIL requires a lot of thought, including thinking about userland app breakage -- a big deal in Linux. I never contended that userspace should be moved away from HCIL. After all, HCIL would be just another label. What I contend is that _internally_ SCSI Core start moving away from HCIL and towards SAM. Most easily this would be done by implementing a bunch of new-way-to-do-it functions. The request_queue wouldn't care, and old LLDD can use the old interface, and new ones can use the new interface. Ask yourself where all these HCIL-addressed CDBs come from... each one of those CDB submittors must be updated from HCIL addressing/routing to transport-specific. No, no transport specific -- I repeat again: the whole point is to move away from transport specific. _SCSI_Core_can invent an HCIL label for them, not LLDD as they currently do. This way LLDD can be more pure and the gunk can go into SCSI Core, _plus_ those same LLDD can now work with the new interface. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote:
On 08/26/05 14:48, Jeff Garzik wrote:
No host numbers, no routing information. This is all
transparent to SCSI Core, and NONE of its business.
Routing is an essential part of the SCSI core's duties.
[I'm not a big fan of reading mixed-message emails, but what can you do...]
The SCSI core is the resource manager responsible for routing messages
[CDBs] to/from LLDs based on scsi-specific device address. This
includes resolution of kernel-specific identifiers (device major/minor,
etc.) into s.s.d.a.. This also includes direct use of
This particular is the task of sd.c. How it does it is
sd.c. job. Not SCSI Core.
No. sd, sr, st, and sg all use the -common- infrastructure to execute
tasks and return results. That common infrastructure is part of the
SCSI core.
The SCSI layer itself is a marraige between
device classes -- sd, sr, st, sg
transport classes -- common per-transport code
drivers -- executes tasks via transport class
glue -- the myriad functions that tie the above 3 together
All transport-specific knowledge that is common across hardware vendors
should be in the transport class. The SCSI core uses the transport
class to perform transport-specific actions.
userspace-provided identifiers as s.s.d.a., such as via SG_IO ioctl.
Ah, yes, I see. So the question is, how do we fit SG_IO ioctl?
SG_IO can be transport/protocol agnostic _if_ SCSI Core gets
the architecture right.
I.e. if I show you a picture of the objects out there in the
SCSI domain, you can just point to one and send something to it.
That picture will be painted by the transport layer. SCSI Core
is _completely_ unaware of all this! This is how you can accomodate
SMP and any future protocols that can come your way.
The SCSI core is the common point for exporting bus topology via
transport classes.
Moving away from HCIL requires a lot of thought, including thinking
about userland app breakage -- a big deal in Linux.
I never contended that userspace should be moved away from HCIL.
Then, by implication, SAS and FC must continue to maintain HCIL-device
maps.
What I contend is that _internally_ SCSI Core start moving
away from HCIL and towards SAM.
SAM is already mostly there. -queuecommand is already a pretty good
execute_task().
Most easily this would be done by implementing a bunch of
new-way-to-do-it functions. The request_queue wouldn't care,
and old LLDD can use the old interface, and new ones can use
the new interface.
Disagree. Just follow the TODO list Christoph outlined, plus figure out
how to handle SG_IO and /dev/sg sanely.
We don't need yet more
if (new way) {
...
} else {
...
}
code blocks :)
HCIL addressing gunk largely belongs in SPI transport class, along with
scsi_scan_host() [each transport class should build its own topology].
This achieves the result you want to achive: new-way-to-do-it functions
live in the SAS and FC transport classes (with generic code in generic
SCSI layer), old-way-to-do-it HCIL functions live in the SPI transport
class.
Ask yourself where all these HCIL-addressed CDBs come from... each one
of those CDB submittors must be updated from HCIL addressing/routing to
transport-specific.
No, no transport specific -- I repeat again: the whole point is to move
away from transport specific. _SCSI_Core_can invent an HCIL label for them,
not LLDD as they currently do.
You will never be able to eliminate transport-specific code. That's the
whole point of transport classes: encapsulate common transport code.
Call it a transport library, rather than a class, if class gives
people the shivers.
Example: I have access to SAS+SATA host controllers from Adaptec and
Company X. Both are largely software-based, directly controlling the
PHY ports and manually performing all SAS+SATA discovery.
I would expect that adaptec_sas and companyx_sas drivers would share
transport-specific code via the ata_transport and sas_transport classes.
Jeff
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote: On 08/26/05 15:24, Jeff Garzik wrote: Luben Tuikov wrote: Even simpler: the transport layer, calls SCSI Core, saying: Hey here is a pointer to struct scsi_domain_device. If you want, you an send REPORT LUNS and other things to it. For the SG_IO ioctl, /dev/sg and request_queue usage, SCSI core must map an address (currently HCIL) into a scsi_domain_device pointer. These The request queue is associated with the LU, not the scsi_domain_device. When SCSI Core discovers the LU, it sets up the request queue for it, etc. Again this is the role of SCSI Core, not messing up with transport specific stuff. upper layer kernel elements rely on this SCSI address, and rely on the fact that SCSI core can route from a block device straight to a SCSI LLD, using nothing more than this SCSI address. I don't get this. More basically... An in-kernel C pointer, to a SCSI target device, is not sufficient in all cases to address a target. This plays out most often in userland interfaces such as ioctls. That is the heart of the routing/addressing that the SCSI core must perform. Disagree: now: scsi_device -- request_queue, then: struct LU -- request_queue. The LU points to the domain_device (as its parent). The domain_device has a void *lldd_dev in it. The current SCSI code largely already has this stuff. Right now the addressing is hardcoded to HCIL. But that can be changed... One proposal was to use (host,string) identifiers. Who? Who is proposing this? I never saw an email to SCSI Core about this proposal? Is there any more information about this proposal and what is the justification of it? Any specs and docs? No specs, just a comment from IRC. (host,string) could succeed in transporting both HCIL and non-HCIL target identifiers. Jeff - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Stefan Richter wrote: Luben Tuikov wanted... to move _away_ from HCIL. So that addressing of devices in SCSI Core is _completely_ independent of addressing at the protocol layer. I second that. A move away from forced HCIL addressing would be a good thing. However, its impossible to completely move away from addressing, as userspace and the SCSI core need ways to route CDBs to devices based on address. Jeff - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Mon, Aug 22, 2005 at 05:08:33PM -0600, Moore, Eric Dean wrote: On Sunday, August 21, 2005 10:53 AM, Christoph Hellwig wrote: This is just a brindup helper because the Fusion hardware does a SAS remote port to target ID mapping in firmware, in fact the firmware interface only addresses them using this assigned ID, which is a big shortcoming in the Fusion interface. Once the more basic things are dealt with that will go away, similarly to how Fusion won't be able to use their firmware mapping for FC once they fully support the FC transport class. Thats incorrect about the Fusion Firmware Interface. The Sas Device Config pages offers three ways to access this config page. They are: (1) GetNextHandle (2) BusTargetID (3) Handle. This is set in config page pageAddr field - bits 28-31. Lets say you want to know all the devices out in the sas domain that is discovered by this HBA, you do the GetNextHandle method. Bits 0-27 is the FormSpecific. In FormSpecific, you pass 0x for the first request, then you pass the devHandle for the previous config page to get the next. The 3.02.55 code I sent you did this discovery in mptbase.c, in a function called mpt_sas_get_info. All this info is stored in a link list, that currently is used for CSMI. That's actually what I'm doing in the patchkit I submitted (please review it an given James a ACK/NACK, thanks), what I meant with addressing was the I/O (or internal frame) submission path, which expects to have the SPI-like target ID set. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Wed, Aug 24, 2005 at 04:05:03PM -0400, Luben Tuikov wrote: On 08/24/05 13:12, Patrick Mansfield wrote: On Tue, Aug 23, 2005 at 07:55:37PM -0400, Luben Tuikov wrote: Where does udev get its label? You can call any script or program from udev and use the result. There are And where does _it_ get it? You can write a program to supply it :) And then try to get it into your favourite distribution ... recent changes for persistent naming, the /dev/by-id naming used in sles9 is in current udev. A tmp /dev node is available to called out programs, so any pass through or regular IO can be sent to the device before it has its real name. So you should be able to get anything available on the scsi device that way. *SCSI DEVICE*, not SAS of FC or iSCSI or whatever. Now, where will that *scsi device* get _its_ label? udev now has an IMPORT rule, see usage of scsi_id, vol_id, etc. in current udev rules. Followup posts should go to [EMAIL PROTECTED] Hey Patrick, is there a point to this thread? Yes, at this juncture a simplified explanation about udev rules. If you have an idea, please go ahead and share and/or implement it. If you're unclear on something please ask it directly. I thought you were asking for details about udev ... your question was rather ambiguous. -- Patrick Mansfield - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
[EMAIL PROTECTED] wrote: - There are some real challenges in supporting a udev-named boot device. For the most part, it's a distro issue, which is becoming better. PS: for $10, name a 2.6 distro that uses udev out of the box for disk names and its installation. For $10 more, can it install/boot from one? Hmm. You are correct in that appearently there _is_ a large education effort required. SLES9 uses udev for installation and booting per default, and this works with even the horribly old version we're having there. - Some of these vendors come from large disk farms, and in several cases, the disk farms change frequently. Thus, they must flush and regenerate their udev configurations on each change. Not a fun process. Not really. You normally shouldn't have to redo the udev configuration as your vendor/distributor will have provided you with one which works out of the box. Of course, you'll have to initiate a rescan of the SAN topology, but udev is the least of the problems you're facing then. - Most could care less about, or don't understand, the technical justifications for the new 2.6 behaviors. Thus, they push their vendors for same-style behaviors as 2.4 regardless of the 2.6 stance. Oh, how true. I heartily agree with you on this one. Cheers, Hannes -- Dr. Hannes Reinecke [EMAIL PROTECTED] SuSE Linux Products GmbHS390 zSeries Maxfeldstraße 5 +49 911 74053 688 90409 Nürnberg http://www.suse.de - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote: On 08/22/05 00:55, Matt Domsch wrote: On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: - There are some real challenges in supporting a udev-named boot device. For the most part, it's a distro issue, which is becoming better. PS: for $10, name a 2.6 distro that uses udev out of the box for disk names and its installation. For $10 more, can it install/boot from one? I won't get your $10, but RHEL4 has a mechanism to specify install onto BIOS disk N, where N is typically 80h (it's an int13 number, what BIOS typically boots from). EDD in anaconda handles the mapping of BIOS disk number to /dev/whatever. Hi Matt! How is it going? I wonder, do you think it would advantageous to provide another label to a disk device which includes this mapping? (This is in lieu to the myriad of labels notion I've been talking about recently, where vendors (BIOS, Dell, disk manufacturer, transport layer, etc, etc, etc.) provide yet another label. This label is tacked to the LU (most commonly a disk, it is *not* an FS label.)) This way, user space, or whoever, can look up the LU (disk device) by a label, that label or whichever one. A note on the subject of LU labels: recent SCSI disks from Fujitsu, Hitachi and Maxtor (but I cannot confirm this for Seagate disks) support the REPORT and SET DEVICE IDENTIFIER SCSI commands. These use a non-volatile area outside a disk's lba space in which a user can write a label (with a max length of 16, 64 (min in current draft) or 512 (max in current draft) bytes depending on the vendor). I believe recent ATA disks also have a similar area where a disk label could be written by the OS/user. Doug Gilbert - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: - the target id is logical for everything but SPI For FC, target ids are typically assigned to devices on a 1st-seen-1st-assigned basis. For several reasons, there can be changes in port discovery order after link events (cable pulls, etc). For 2.4 kernels, the driver typically maintained a mapping within the driver to ensure the same device showed up at the same target id, even if it disappeared for some amount of time. If FC was the boot device, wacky methods were used to ensure the mapping persisted boot-to-boot. As others stated, id is already a tag/label. You should be able to pass whatever id you want to scsi_scan_target, like the FC ID (port_id), and then we also want an abstract iterator in fc transport for the id for usage in scsi_scan.c:scsi_scan_channel. Then you can lose all the fc_host-next_target_id code. Printing of them won't be so nice, we should have used hex values in our device's bus_id. Hex would also make scsi class/bus_id shorter. We can in theory overflow it today: (5 digits per short + 10 digits per int * 3 + 3 :'s) = 38 But BUS_ID_SIZE (KOBJ_NAME_LEN) is 20. 8 byte lun would add on another 10 bytes (if in decimal). We generally don't have problems since channel is one digit (it should be a char or bit), id is 100, and host 1000 ... lun I'm not sure about: (3 + 1 + 2 + 10) + 3 = 19. i.e. use port_id as the id (8 digits?) and we are more likely to go above 20. -- Patrick Mansfield - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
As others stated, id is already a tag/label. You should be able to pass whatever id you want to scsi_scan_target, like the FC ID (port_id), and then we also want an abstract iterator in fc transport for the id for usage in scsi_scan.c:scsi_scan_channel. Then you can lose all the fc_host-next_target_id code. All nice and well, but How did this help things ? The issue was the device with a changing target id. If the device comes back at the same address each and every time, ok. But, with FC, the Port ID is temporal. It can change on a loop init, fabric reconfig, or with a user cable swap (kicked the cable and replugged). If the port id changes during run time, what are you to do ? What if a new port is seen at the old port id, how do you now deal with the name conflict ? You know apps are going to key off the physical bus address being the target - if it changes, this becomes very problematic. This approach only works as long as the transport's id fits within the target id (an int). How would the SAS address(8byte wwn) utilize such a scheme ? -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
[EMAIL PROTECTED] wrote: As others stated, id is already a tag/label. You should be able to pass whatever id you want to scsi_scan_target, like the FC ID (port_id), [...] If the port id changes during run time, what are you to do ? [...] This approach only works as long as the transport's id fits within the target id (an int). How would the SAS address(8byte wwn) utilize such a scheme ? I don't understand exactly what is being discussed here. But if anybody's intention is to extend the concept of target ID to also provide the concept of transport specific, persistent, globally unique identifiers*, then the datatype of target ID needs to be extended accordingly. Otherwise the LLDs have to implement a mapping between target IDs and these unique identifiers. This mapping does not need to be persistent. Persistent mappings for whatever purpose can and should be implemented in userspace. (This is how it has always been e.g. with SBP-2.) *) examples of transport specific, persistent, globally unique identifiers: obviously 8 Byte address from SAS, 8 Byte EUI from SBP-2/ IEEE 1394/ ISO/IEC 13213, or maybe 88 bit GUI from ISO/IEC 13213 -- Stefan Richter -=-=-=-= =--- =-=== http://arcgraph.de/sr/ - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
I thought by the target id is logical for everything but SPI you meant that FC enumerated the scsi_device id. Yes, I meant that. I didn't mean to address problems with persistent names, just map the scsi_device id to an FC value. True. Port ID is just a bad example. Unfortunately, there's nothing in FC, outside of wwpn (and sometimes wwnn), that you can uniquely identify a device by. Since the wwpn didn't fit in the scsi_device id, it mandated a translation to something that did. A simple sequential counter was chosen. To keep things consistent, and done in a LLDD-independent manner, the FC transport owned the assignment of the counter and the persistency (and consistency) of the mapping. -- james - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/23/05 13:28, Stefan Richter wrote: [EMAIL PROTECTED] wrote: As others stated, id is already a tag/label. You should be able to pass whatever id you want to scsi_scan_target, like the FC ID (port_id), [...] If the port id changes during run time, what are you to do ? [...] This approach only works as long as the transport's id fits within the target id (an int). How would the SAS address(8byte wwn) utilize such a scheme ? I don't understand exactly what is being discussed here. But if anybody's intention is to extend the concept of target ID to also provide the concept of transport specific, persistent, globally unique identifiers*, then the datatype of target ID needs to be extended accordingly. Hi Stefan, No, no, no -- this is exactly what we DO NOT WANT TO DO. Why? Because it is the same mistake that was done when HCIL crept up into SCSI Core 13 years ago. Device addressing done by LLDD is completely not the business of SCSI Core and vice versa. They should be *completely* separate. In effect, a native support for targets (I know... a poor choice of words, but not everyone reads specs), has been in need for SCSI Core since one of the first iSCSI Target and Initiator for Linux was developed 5 years ago. In effect FC tells SCSI Core: here is a device I do not know anything about, other than the fact that it is on the fabric and it has a SCSI Target (or Initiator) Port: how you get to is is _my_ business, what is done with or to the device is _your_ business. Then SCSI Core does LU discovery using SPC and SAM on the device using Execute Task. All this _completely_ eliminates the need for HCIL. How the FC layer identifies the device is the FC's layer problem. How SCSI Core identifies the device is SCSI Core's problem. HCIL should have never been available to SCSI Core in the first place, but then again SAM didn't exist back then. Otherwise the LLDs have to implement a mapping between target IDs and LLDD should not have to invent HCIL just because SCSI Core hasn't seen any inovation and advancement for the last 13 years. Currently they do. Target ID should NOT be extended to cover whatever new transport protocols comes up. SCSI Core whould follow SAM if it wants to transparently cover FC, iSCSI, SAS, etc. these unique identifiers. This mapping does not need to be persistent. Persistent mappings for whatever purpose can and should be implemented in userspace. (This is how it has always been e.g. with SBP-2.) *) examples of transport specific, persistent, globally unique identifiers: obviously 8 Byte address from SAS, 8 Byte EUI from SBP-2/ IEEE 1394/ ISO/IEC 13213, or maybe 88 bit GUI from ISO/IEC 13213 Yes, you're right. And if would be nice if all those IDs are just another label tacked onto the domain device, opaque label. SCSI Core can add labels on its own (e.g. HCIL ;-) ). Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/23/05 11:42, Patrick Mansfield wrote: On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: - the target id is logical for everything but SPI For FC, target ids are typically assigned to devices on a 1st-seen-1st-assigned basis. For several reasons, there can be changes in port discovery order after link events (cable pulls, etc). For 2.4 kernels, the driver typically maintained a mapping within the driver to ensure the same device showed up at the same target id, even if it disappeared for some amount of time. If FC was the boot device, wacky methods were used to ensure the mapping persisted boot-to-boot. As others stated, id is already a tag/label. You should be able to pass whatever id you want to scsi_scan_target, like the FC ID (port_id), and No! The whole point is to move _away_ from HCIL. So that addressing of devices in SCSI Core is _completely_ independent of addressing at the protocol layer. In fact SCSI Core neither needs to address/enumerate the devices, nor it needs to know any kind of addressing. All SCSI Core needs is a pointer to a structure that it can pass to the LLDD's Execute Task, so that the task can be sent to that device and executed. This allows great versatility between type of device, protocol it speaks, command set it uses, transport used to talk to the device. then we also want an abstract iterator in fc transport for the id for usage in scsi_scan.c:scsi_scan_channel. Then you can lose all the fc_host-next_target_id code. This gets too complicated and unnecessary. First there is no such thing as scsi_scan_channel() for FC or for SAS or for iSCSI or for ... If you use propely the kobject model, no such container is needed. Hotplug works very nice too. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/22/05 00:55, Matt Domsch wrote: On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: - There are some real challenges in supporting a udev-named boot device. For the most part, it's a distro issue, which is becoming better. PS: for $10, name a 2.6 distro that uses udev out of the box for disk names and its installation. For $10 more, can it install/boot from one? I won't get your $10, but RHEL4 has a mechanism to specify install onto BIOS disk N, where N is typically 80h (it's an int13 number, what BIOS typically boots from). EDD in anaconda handles the mapping of BIOS disk number to /dev/whatever. Hi Matt! How is it going? I wonder, do you think it would advantageous to provide another label to a disk device which includes this mapping? (This is in lieu to the myriad of labels notion I've been talking about recently, where vendors (BIOS, Dell, disk manufacturer, transport layer, etc, etc, etc.) provide yet another label. This label is tacked to the LU (most commonly a disk, it is *not* an FS label.)) This way, user space, or whoever, can look up the LU (disk device) by a label, that label or whichever one. And a few folks at Dell are working on a udev helper to let udev know this mapping too, and incorporating the same capability into SLES in the future. After install, file system labels have been working great for years. root=LABEL=/ syntax for example, and you can get as complex as you like with that label. Yes, that's cool. How about specifying the LU label _and_ the FS label to root=? root=[disk label:]fs label disc label := WWN, HCIL, Funky name 1, Funky name 2. fs label := filesystem specific label. All those labels are tacked onto the LU, and are provided by various layers: transport, LLDD, vendor, manufacturer, etc, etc, etc. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: - There are some real challenges in supporting a udev-named boot device. For the most part, it's a distro issue, which is becoming better. PS: for $10, name a 2.6 distro that uses udev out of the box for disk names and its installation. For $10 more, can it install/boot from one? I won't get your $10, but RHEL4 has a mechanism to specify install onto BIOS disk N, where N is typically 80h (it's an int13 number, what BIOS typically boots from). EDD in anaconda handles the mapping of BIOS disk number to /dev/whatever. And a few folks at Dell are working on a udev helper to let udev know this mapping too, and incorporating the same capability into SLES in the future. After install, file system labels have been working great for years. root=LABEL=/ syntax for example, and you can get as complex as you like with that label. Thanks, Matt -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
--- Christoph Hellwig [EMAIL PROTECTED] wrote: For SAS, looking at Christoph's code - The target id comes from the LLDD. This is just a brindup helper because the Fusion hardware does a SAS remote port to target ID mapping in firmware, in fact the firmware interface only addresses them using this assigned ID, which is a big shortcoming in the Fusion interface. Once the more basic things are :-) dealt with that will go away, similarly to how Fusion won't be able to use their firmware mapping for FC once they fully support the FC transport class. You should not deny the possibility of vendors providing HCIL or any other (kind of) label for the LU. However it is possible to move into the 21st century _and_ provide HCIL or any other (kind of) label to anyone who's interested. Vendors would want to name the LU something, so that it is identifiable to end users. Vendors may want to provide one or many labels. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Sat, Aug 20, 2005 at 12:15:41AM -0400, [EMAIL PROTECTED] wrote: Now - back to Christoph's point For iSCSI, (correct me if I'm wrong) as the scsi_host is mapped 1:1 with the session, there really is no such thing as multiple targets, so it works. Even if it isn't 1:1, it is controlled via user space so it's likely managable. Note: instead of target id shifting, there may now be scsi_host number shifting. That's the case for the open-iscsi (and the older cisco) software initiator. For the two iSCSI HBA drivers I've seen it's very different. For SAS, looking at Christoph's code - The target id comes from the LLDD. This is just a brindup helper because the Fusion hardware does a SAS remote port to target ID mapping in firmware, in fact the firmware interface only addresses them using this assigned ID, which is a big shortcoming in the Fusion interface. Once the more basic things are dealt with that will go away, similarly to how Fusion won't be able to use their firmware mapping for FC once they fully support the FC transport class. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Sat, Aug 20, 2005 at 01:34:49PM -0400, Luben Tuikov wrote: that even if my minimal code goes in now there's absolutely no reason we can't replace it completely later on. See the evolution of the FC transport class. Who makes all those decisions? In the end James, unless he'd be over-ruled by Linus (which is rather unlikely) More generally, why is SCSI Core not being managed by Documentation/ManagingStyle? It is. Is it because there's so much vendor interest here? There's about as much vendor interest as elsewhere. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Sat, Aug 20, 2005 at 01:23:04PM -0400, Luben Tuikov wrote: I don't mind LLDD giving channel and id to SCSI Core. Not at all. What I mind is the _way_ it is done. Just consider this (and I'm sure you have the same sentiments): slave alloc gives you channel and id and lun/2 to find out the device it wants to poke at... And the really sad part is that NO ONE at linux-scsi finds this objectionable. This should've been thrown out 5 years ago (well slave alloc wasn't around then) when iSCSI was making its way in, and when people suggested it. Sadly it was shot down by the Maintainers and this is what we have here today. -slave_alloc gives you a scsi_device. With proper transport class integration that can include lots of transport-specific information. Ok, to answer both your and Jeff's email, this is how this all is done: Let, (channel, id) tuple be *just another label*! They pretty much are that as far as the scsi core code is concerned. Just do a little grep for -channel and -id over the scsi core, there's very few users except printks, and most of those few are in optional library functions that should move into the SPI transport class one day. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Fri, Aug 19, 2005 at 04:32:15PM -0400, Luben Tuikov wrote: The current SAS class will only get validated when it actually meets real SAS topologies, which is acceptable in my view just to get this project actually moving; code can always be updated later ... James, the current SAS class _will_go_ into the kernel because: - It is 3 vendor driven: LSI, Dell, HP. - It is being developed by you and Christoph, the people who decide what goes in or not. No, it will go in because it's the only class actually available. I'd still love to see any code from you posted publically. I've been forwarded in private some code you sent around to a few people at OLS, but you still can't be bothered to actually posting it publically. Not that even if my minimal code goes in now there's absolutely no reason we can't replace it completely later on. See the evolution of the FC transport class. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/20/05 00:15, [EMAIL PROTECTED] wrote: Ok, so I guess I owe you a response. I hesitate, as this discussion always becomes larger than it should. There are 2 key points: - the target id is logical for everything but SPI - following old SPI behavior, users expect the same devices to be enumerated at the same target ids, regardless of whether that enumeration is at the next link event, driver load/unload, or system reboot. The corollary for this is that the device's name should have remained the same as well. I don't mind LLDD giving channel and id to SCSI Core. Not at all. What I mind is the _way_ it is done. Just consider this (and I'm sure you have the same sentiments): slave alloc gives you channel and id and lun/2 to find out the device it wants to poke at... And the really sad part is that NO ONE at linux-scsi finds this objectionable. This should've been thrown out 5 years ago (well slave alloc wasn't around then) when iSCSI was making its way in, and when people suggested it. Sadly it was shot down by the Maintainers and this is what we have here today. For FC, target ids are typically assigned to devices on a 1st-seen-1st-assigned basis. For several reasons, there can be changes in port discovery order after link events (cable pulls, etc). For 2.4 kernels, the driver typically maintained a mapping within the driver to ensure the same device showed up at the same target id, even if it disappeared for some amount of time. If FC was the boot device, wacky methods were used to ensure the mapping persisted boot-to-boot. This is all _fine_ and I don't mind it at all. For SAS, looking at Christoph's code - The target id comes from the LLDD. So either the LLDD maintains a map of SAS port addresses to target ids, or the mapping could change, same as FC. Christoph's argument is that FC's issue was historical. SAS is sufficiently new that folks are now smart enough to use udev. My contention is that people will expect the same behavior out of SAS as they did w/ FC. To them its all just SCSI. Ok, to answer both your and Jeff's email, this is how this all is done: Let, (channel, id) tuple be *just another label*! For a domain device or LU in a domain device you can have a myriad of labels, depending on a myriad of things, like - how you got to the device - what transport you used to get the device - what kind of device it actually is, - what kind of interrogation methods it replies to, - what kind of VPD it has if any, - what the Command Set driver named it (sdX, stX), - etc. Now, the function of a LLDD is to provide access to the transport, this is done by the LLDD implementing SAM's Execute Task and the TMFs. Next, the transport layer uses those to interrogate the domain the LLDD provides access to and find out any domain devices. Each domain device is then registered with SCSI Core, using something like a struct domain_device. Now _that_ struct domain_device can have as many and as varied _labels_ as you want: (channel, id), WWN, Funky name1, Funky name2, name given by the Command Set driver, which as kept in the struct domain_device. Then you expose _those_ labels to user space and let programs use whatever label they want. The kernel itself can use the struct domain device to address the device, but anyone else from user space can use one of the labels: WWN, (channel,id), etc. This will give you backward compatibility, plus it would allow for SCSI Core to be completely rewritten in the spirit of SAM and enter the 21 century. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/20/05 05:18, Christoph Hellwig wrote: On Fri, Aug 19, 2005 at 04:32:15PM -0400, Luben Tuikov wrote: The current SAS class will only get validated when it actually meets real SAS topologies, which is acceptable in my view just to get this project actually moving; code can always be updated later ... James, the current SAS class _will_go_ into the kernel because: - It is 3 vendor driven: LSI, Dell, HP. - It is being developed by you and Christoph, the people who decide what goes in or not. No, it will go in because it's the only class actually available. I'd still love to see any code from you posted publically. I've been forwarded in private some code you sent around to a few people at OLS, Thats good, I was hoping that you'd get it. but you still can't be bothered to actually posting it publically. Not C'mon Christoph -- no one more than me wants to see SCSI Core improved. 5 years ago because of iSCSI, now because of SAS. It's not about being bothered, it's just that it's not quite finished yet. that even if my minimal code goes in now there's absolutely no reason we can't replace it completely later on. See the evolution of the FC transport class. Who makes all those decisions? More generally, why is SCSI Core not being managed by Documentation/ManagingStyle? Is it because there's so much vendor interest here? Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Thu, Aug 18, 2005 at 03:04:01PM -0400, Jeff Garzik wrote: Depending on how SATA support is implemented, it may be useful for that. You can already see how SATA support is implemented ;-) We just need to wrap it in a transport class. SATA support for SAS host adapters. Especially when talking STP to a bridge this is a little more complicated than the current libata model. That doesn't mean I don't want to share code and mechanisms but it'll need a little refactoring at least. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Thu, Aug 18, 2005 at 04:05:29PM -0400, Luben Tuikov wrote: On 08/18/05 13:56, Christoph Hellwig wrote: I was trying to avoid the need for an rport object, but I'm not yet sure it's feasible. We certainly won't put in target-persistency support similar to FC, that was just a hack to get the existing drivers migrated without lots of complaints, it's not going in for new transports like SAS or iSCSI. Hmm, I remember working on iSCSI exactly 5 years ago. Is this considered new? (er, I mean in light of SCSI Core) We've just added the first iSCSI LLDD (the open-iscsi software initiator) to the scsi git trees, so yes, this is considered new. I also remember it was that time when I asked about 8 byte LUNs to be supported transparently by SCSI Core. It was also that time when I asked for target to be supported without channel and id as this had to be invented by the iSCSI LLDD as it needs to be invented by FC and USB. Or just ignored. What we might need an rport for is to support SMP. I'm not yet sure how to do SMP passthrough, but we will need some object to represent SMP ports. Why? How is this going to be used? What is the architecture model? We need some way to implement SMP passthrough. One of the options is to have a passthrough node that frames can be written to - for that we obviously need an object for non-scsi target sas remote ports. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/19/05 10:06, Christoph Hellwig wrote: SATA support for SAS host adapters. Especially when talking STP to a bridge this is a little more complicated than the current libata model. That doesn't mean I don't want to share code and mechanisms but it'll need a little refactoring at least. I'm not understanding what you mean here. If the lower level is implemented correctly, libata would just pick up and drive the ATA device. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Fri, Aug 19, 2005 at 01:51:56PM -0400, Luben Tuikov wrote: On 08/19/05 10:06, Christoph Hellwig wrote: SATA support for SAS host adapters. Especially when talking STP to a bridge this is a little more complicated than the current libata model. That doesn't mean I don't want to share code and mechanisms but it'll need a little refactoring at least. I'm not understanding what you mean here. If the lower level is implemented correctly, libata would just pick up and drive the ATA device. Yes. But to get that to work we'll need to fix some assumptions in libata, define proper interfaces between the SAS LLDD, the SAS transport class and libata. It's doable and desirable, but it'll need some work. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/19/05 13:54, Christoph Hellwig wrote: On Fri, Aug 19, 2005 at 01:51:56PM -0400, Luben Tuikov wrote: On 08/19/05 10:06, Christoph Hellwig wrote: SATA support for SAS host adapters. Especially when talking STP to a bridge this is a little more complicated than the current libata model. That doesn't mean I don't want to share code and mechanisms but it'll need a little refactoring at least. I'm not understanding what you mean here. If the lower level is implemented correctly, libata would just pick up and drive the ATA device. Yes. But to get that to work we'll need to fix some assumptions in libata, define proper interfaces between the SAS LLDD, the SAS transport class and libata. It's doable and desirable, but it'll need some work. Do you have a SAS LLDD? Which one is it that will let you do this? Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Fri, Aug 19, 2005 at 01:56:39PM -0400, Luben Tuikov wrote: Yes. But to get that to work we'll need to fix some assumptions in libata, define proper interfaces between the SAS LLDD, the SAS transport class and libata. It's doable and desirable, but it'll need some work. Do you have a SAS LLDD? See the mptsas driver I posted today. Which one is it that will let you do this? Currently none. Fusion is doing the SATA command translation in firmware. This is more for Adapters in the style of the Adaptec design where it would be done in the host software stack. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/19/05 13:59, Christoph Hellwig wrote: Which one is it that will let you do this? Currently none. Fusion is doing the SATA command translation in firmware. This is more for Adapters in the style of the Adaptec design where it would be done in the host software stack. So you're doing architectural decisions based on guesses on how Adaptec's (design) driver is? Why not let Jeff worry about SATA stuff? There is a lot more areas where SCSI core needs improvement -- *that* would be commendable work. Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/19/05 13:59, Christoph Hellwig wrote: On Fri, Aug 19, 2005 at 01:56:39PM -0400, Luben Tuikov wrote: Yes. But to get that to work we'll need to fix some assumptions in libata, define proper interfaces between the SAS LLDD, the SAS transport class and libata. It's doable and desirable, but it'll need some work. Do you have a SAS LLDD? See the mptsas driver I posted today. Sorry, this is _not_ a SAS LLDD. There is *nothing*, I repeat *nothing* SAS in that code. Luben Which one is it that will let you do this? Currently none. Fusion is doing the SATA command translation in firmware. This is more for Adapters in the style of the Adaptec design where it would be done in the host software stack. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On Fri, 2005-08-19 at 14:07 -0400, Luben Tuikov wrote: So you're doing architectural decisions based on guesses on how Adaptec's (design) driver is? You can't have it both ways. We have to take a fully theoretical approach (which does involve guesswork) because only the vendors have the actual silicon and devices to set up a SAS topology. However, it has become equally clear that we cannot rely on the vendors to come up with a SAS class (and this in not for want of effort on our part). The current SAS class will only get validated when it actually meets real SAS topologies, which is acceptable in my view just to get this project actually moving; code can always be updated later ... There is a lot more areas where SCSI core needs improvement -- *that* would be commendable work. Patches are always welcome as long as they solve real problems or make real improvements. Open source is itch driven to a large extent (as in if something bites you, you have the impetus to scratch the sore place and fix it) ... remember society doesn't encourage the scratching of other peoples itches (in public at least) ... James - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/19/05 15:59, James Bottomley wrote: On Fri, 2005-08-19 at 14:07 -0400, Luben Tuikov wrote: So you're doing architectural decisions based on guesses on how Adaptec's (design) driver is? You can't have it both ways. We have to take a fully theoretical You just made a decision. approach (which does involve guesswork) because only the vendors have the actual silicon and devices to set up a SAS topology. However, it You mean, only the vendor engineers who fall asleep with a bunch of 400 page specs all over their bed. has become equally clear that we cannot rely on the vendors to come up with a SAS class (and this in not for want of effort on our part). You just made your position clear. The current SAS class will only get validated when it actually meets real SAS topologies, which is acceptable in my view just to get this project actually moving; code can always be updated later ... James, the current SAS class _will_go_ into the kernel because: - It is 3 vendor driven: LSI, Dell, HP. - It is being developed by you and Christoph, the people who decide what goes in or not. Who cares if it is validated by real SAS topoligies? You can _tweak_ it to do so, until you make it unmanageable. Who cares about the technology and/or if it is anything close to what the specs say? Who cares if the LLDD has nothing to do with the technology or SAS? I mean after all, the firmware does SAS, not the LLDD, not a SAS layer, not SCSI Core. The state of SCSI Core is that it was developed mimicking SPI, many many years ago, after that there hasn't been any _SCSI_storage_ improvements, other than local inventions. All the while technologies come (and go): FC, iSCSI, USB Storage, SATA, SAS and SCSI Core stays pretty much the same. There is a lot more areas where SCSI core needs improvement -- *that* would be commendable work. Patches are always welcome as long as they solve real problems or make real improvements. Open source is itch driven to a large extent (as Well, as far as I remember it wasn't until Andi Kleen who asked if anyone has tried scsi command allocation from a slab cache that I posted my results for a _second_ time. Then thanks to Doug L. they went in, along with scsi_done_q. Everthing is subjective -- your definition of real problem, real improvements and solving would differ from someone else's. It's all about our point of view. SCSI Core develops differently from say, the MM subsystem. The reason, obious to everyone, is that there is so much more vendor interest in here. in if something bites you, you have the impetus to scratch the sore place and fix it) ... remember society doesn't encourage the scratching of other peoples itches (in public at least) ... Thank you for letting everyone know your position, Luben James - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Luben Tuikov wrote: James, the current SAS class _will_go_ into the kernel because: - It is 3 vendor driven: LSI, Dell, HP. - It is being developed by you and Christoph, the people who decide what goes in or not. No, not for these reasons. It will go in because it is small, and can be improved over time as vendors such as Adaptec teach us the proper topology. It is a standard problem: engineers know their hardware, and know the specs, but not the Linux development process. An Adaptec SAS driver can be in the 2.6.14 kernel, if it is small. The process then starts: 1) Luben points out a problem. 2) The best solution is discussed. 3) A patch is created to solve that one problem. 4) Go to step #1. We grow SAS support, like growing a plant or tree. Start with a seed, not a full-grown tree. Jeff - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
This is a heck of a statement... The customers don't see it as SAS vs FC vs SPI, they just see it as SCSI, and there's a lot of expectations on consistent behavior. We take a lot of heat defending the community's position, even from companies that you would have thought had signed on to the 2.6 behaviors. I tried to understand this paragraph in the light of the very few past days, but I couldn't. What is it actually saying? I understand the need to push folks to the new 2.6 models, but I fully expect the same complaints to come from the users of SAS and iSCSI as well. What kind of complaints? I ask because I'm a user of SAS and iSCSI. Ok, so I guess I owe you a response. I hesitate, as this discussion always becomes larger than it should. There are 2 key points: - the target id is logical for everything but SPI - following old SPI behavior, users expect the same devices to be enumerated at the same target ids, regardless of whether that enumeration is at the next link event, driver load/unload, or system reboot. The corollary for this is that the device's name should have remained the same as well. For FC, target ids are typically assigned to devices on a 1st-seen-1st-assigned basis. For several reasons, there can be changes in port discovery order after link events (cable pulls, etc). For 2.4 kernels, the driver typically maintained a mapping within the driver to ensure the same device showed up at the same target id, even if it disappeared for some amount of time. If FC was the boot device, wacky methods were used to ensure the mapping persisted boot-to-boot. For 2.6 kernels, the desire is to move away from relying on the physical addressing. The recommendation is to use udev (preferred) or filesystem labels (ok in some circumstances) to find the right devices. By moving to lun/device-level id's, issues such as name shift are better solved (note: name-shift still existed under 2.4). We have found users having difficulty with the udev transition, as: - From 2.4 history, old scsi behavior, and other unix's behavior (which functions much like 2.4), they are accustomed to not needing external tools or admin steps for device nameing. Things had just worked for the most part. - Most view 2.6 as an upgrade and didn't expect something this basic to change. - Many really don't know about or understand udev, hotplug, disk identifiers, and so on. Thus, there's a large education effort. It has to be dealt with in yet more documentation, help lines, etc. - There are some real challenges in supporting a udev-named boot device. For the most part, it's a distro issue, which is becoming better. PS: for $10, name a 2.6 distro that uses udev out of the box for disk names and its installation. For $10 more, can it install/boot from one? - For better or worse, tools and api's exist that interacted with the old 2.4-style behavior. Now they must wait for the tools to be updated, suffer their non-functionality, and/or craft their own tools. - Some of these vendors come from large disk farms, and in several cases, the disk farms change frequently. Thus, they must flush and regenerate their udev configurations on each change. Not a fun process. - Most could care less about, or don't understand, the technical justifications for the new 2.6 behaviors. Thus, they push their vendors for same-style behaviors as 2.4 regardless of the 2.6 stance. Now - back to Christoph's point For iSCSI, (correct me if I'm wrong) as the scsi_host is mapped 1:1 with the session, there really is no such thing as multiple targets, so it works. Even if it isn't 1:1, it is controlled via user space so it's likely managable. Note: instead of target id shifting, there may now be scsi_host number shifting. For SAS, looking at Christoph's code - The target id comes from the LLDD. So either the LLDD maintains a map of SAS port addresses to target ids, or the mapping could change, same as FC. Christoph's argument is that FC's issue was historical. SAS is sufficiently new that folks are now smart enough to use udev. My contention is that people will expect the same behavior out of SAS as they did w/ FC. To them its all just SCSI. I'm winded. Hope this helps. -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
Christoph, I like it, and have no real complaints. As familiar as this looks, there were a couple of conventions in the FC transport that I thought should have carried over here. Namely, I saw not all attributes being the same, thus I created 3 categories of attributes: Private: These are attributes fully owned by the transport. The LLDD does not directly access them, or participate in the sysfs calls. LLDD interaction is strictly indirect via transport functions. Fixed: These attributes, once set, are not expected to change. The LLDD does set these values directly, but should only do so at initialization. The sysfs functions will be handled solely by the transport w/ no interaction with the LLDD. Dynamic (Normal): Values can change. Sysfs functions utilize LLDD to get/set values. I expect that the SAS transport has much the same categories, and it would be beneficial to indicate which category the different attributes fall into. This can be as simple as comments, grouping, or name prefixes. Also, I see your enums set explicit values. Just a caution. When I did the FC transport, there were cases where I specifically did not use specification-specific values, as the transport was a subset or extension of the spec. This made anyone interacting with the transport realize that this area was not just a reincarnation of the spec (plus, spec writers sometimes do stupid things). -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
Ok, so I'm going to have to revert my statement. After perusing the sas transport, I went to the previous patch, which added pre-init data to scsi_scan_target calls. I didn't understand why this was needed. In tracking how you were using the patch, it highlighted that you were skipping a step. The api is such that it does not expect remote SAS ports to be instantiated. They should have be (just like FC). Your api is written only to instantiate local initiator SAS ports. It needs to instantiate remote SAS ports as well. If it does so, the remote port is the parent, and the pre-init data doesn't need to be passed around. The remote SAS ports need to also implement consistent target id bindings, just like FC. Also, it appears that your api is allowing multiple SAS initiator ports to SCSI host. This doesn't make sense, unless you are going to map SAS ports to the scsi channel designator in the address. In FC, there is a 1:1 correlation of the scsi_host to the local FC port, so there's no need for a local port transport object as it's simply the transport for the host. Also, I prefer a distinction between the local port and remote port as the functionality of each will be different (e.g. for local port, you'll have statistics, more attributes, and more functions that can be performed. The remote ports are rarely more than a reporting element, so they are less featured). -- James -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Smart, James Sent: Thursday, August 18, 2005 7:58 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; linux-scsi@vger.kernel.org Subject: RE: [PATCH] minimal SAS transport class Christoph, I like it, and have no real complaints. As familiar as this looks, there were a couple of conventions in the FC transport that I thought should have carried over here. Namely, I saw not all attributes being the same, thus I created 3 categories of attributes: Private: These are attributes fully owned by the transport. The LLDD does not directly access them, or participate in the sysfs calls. LLDD interaction is strictly indirect via transport functions. Fixed: These attributes, once set, are not expected to change. The LLDD does set these values directly, but should only do so at initialization. The sysfs functions will be handled solely by the transport w/ no interaction with the LLDD. Dynamic (Normal): Values can change. Sysfs functions utilize LLDD to get/set values. I expect that the SAS transport has much the same categories, and it would be beneficial to indicate which category the different attributes fall into. This can be as simple as comments, grouping, or name prefixes. Also, I see your enums set explicit values. Just a caution. When I did the FC transport, there were cases where I specifically did not use specification-specific values, as the transport was a subset or extension of the spec. This made anyone interacting with the transport realize that this area was not just a reincarnation of the spec (plus, spec writers sometimes do stupid things). -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
One more thing missing... As there is a transport object between the scsi_host and scsi_target, the transport needs to pick up the changes made to deal with scans made outside of the transport (after a scsi_target has been removed). The fix that got merged into the scsi-misc tree can be seen at: http://www.kernel.org/pub/linux/kernel/people/jejb/scsi-rc-fixes-2.6.changelog http://www.kernel.org/pub/linux/kernel/people/jejb/scsi-rc-fixes-2.6.diff and correspond to the patch at: http://marc.theaimsgroup.com/?l=linux-scsim=111845669410785w=2 -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] minimal SAS transport class
I was trying to avoid the need for an rport object, but I'm not yet sure it's feasible. We certainly won't put in target-persistency support similar to FC, that was just a hack to get the existing drivers migrated without lots of complaints, it's not going in for new transports like SAS or iSCSI. This is a heck of a statement... The customers don't see it as SAS vs FC vs SPI, they just see it as SCSI, and there's a lot of expectations on consistent behavior. We take a lot of heat defending the community's position, even from companies that you would have thought had signed on to the 2.6 behaviors. I understand the need to push folks to the new 2.6 models, but I fully expect the same complaints to come from the users of SAS and iSCSI as well. Please note that by implementing the consistent mappings, you are lessening the demands on the hba vendor to supply a non-upstream driver that works around the issue. What we might need an rport for is to support SMP. I'm not yet sure how to do SMP passthrough, but we will need some object to represent SMP ports. Depending on how SATA support is implemented, it may be useful for that. -- james s - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/18/05 13:56, Christoph Hellwig wrote: I was trying to avoid the need for an rport object, but I'm not yet sure it's feasible. We certainly won't put in target-persistency support similar to FC, that was just a hack to get the existing drivers migrated without lots of complaints, it's not going in for new transports like SAS or iSCSI. Hmm, I remember working on iSCSI exactly 5 years ago. Is this considered new? (er, I mean in light of SCSI Core) I also remember it was that time when I asked about 8 byte LUNs to be supported transparently by SCSI Core. It was also that time when I asked for target to be supported without channel and id as this had to be invented by the iSCSI LLDD as it needs to be invented by FC and USB. It was also that time when I asked for target to be represented transparently so that SCSI Core can do LU discovery and register the devices found and register them with the Command Set Drivers (sd, st, etc). On all three points I saw great opposition from the maintainers. Sadly, I had implemented all those things, but the products (target and initiator) was closed source. This is where scsi_done_q and scsi commands from slab cache came from... What we might need an rport for is to support SMP. I'm not yet sure how to do SMP passthrough, but we will need some object to represent SMP ports. Why? How is this going to be used? What is the architecture model? Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
On 08/18/05 14:48, [EMAIL PROTECTED] wrote: This is a heck of a statement... The customers don't see it as SAS vs FC vs SPI, they just see it as SCSI, and there's a lot of expectations on consistent behavior. We take a lot of heat defending the community's position, even from companies that you would have thought had signed on to the 2.6 behaviors. I tried to understand this paragraph in the light of the very few past days, but I couldn't. What is it actually saying? I understand the need to push folks to the new 2.6 models, but I fully expect the same complaints to come from the users of SAS and iSCSI as well. What kind of complaints? I ask because I'm a user of SAS and iSCSI. Please note that by implementing the consistent mappings, you are lessening the demands on the hba vendor to supply a non-upstream driver that works around the issue. So LLDD would still have to do wild scsi transport X domain to SPI Centric HCIL used by SCSI Core translation? I'm not quite understanding. What we might need an rport for is to support SMP. I'm not yet sure how to do SMP passthrough, but we will need some object to represent SMP ports. Depending on how SATA support is implemented, it may be useful for that. Why do you need SMP ports? Why and how are they going to be used for SATA? Thanks, Luben - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] minimal SAS transport class
Good stuff, Christoph!
Luben
--- Christoph Hellwig [EMAIL PROTECTED] wrote:
This is a minmal, bottom-up SAS transport class. So far it only exposed
information about the SAS port/phy and SAS device (scsi target). I hope
this will integrate nicely with the top-down work Luben has done once he
finally releases it publically, but for now I think we should have
something so SAS drivers can go in the tree.
In detail this transport class does:
- introduces a SAS port object between the Scsi_Host and scsi_target,
this is used to hold all information specific to the SAS port and
PHY - right now they're used interchangable as I haven't found the
right abstraction for wide ports yet - if there is a proper solution
at all as the SAS spec leaves binding PHYs together to wide ports
up to the implementation.
- adds some attributes to the scsi_target, and an API call to
preinitialize them.
It does not:
- handle any managment interfaces or chaning of attributes
- any SAS devices that are not scsi targets, most importantly there's
no support for SMP and extenders yet
- wide ports (as mentioned above)
- software device discovery (although I know Luben has some nice code
for that)
- everythig not mentioned here
A bit of warning: I've only tested this with an SATA disk attached to
a SAS HBA so far because I don't have any real SAS storage yet.
To use the transport class you need a patched fusion driver for now,
use the LSI tarball at:
ftp://ftp.lsil.com/HostAdapterDrivers/linux/Fusion-MPT/mptlinux-3.02.55-src.tar.gz
plus my patch at:
http://verein.lst.de/~hch/fusion-sas-transport-class.diff
I'll try to port my changes plus basic SAS support over to the mainline
driver, but the driver is currently not endian clean which makes it hard
for me to actually test it.
Signed-off-by: Christoph Hellwig [EMAIL PROTECTED]
Index: scsi-misc-2.6/drivers/scsi/Kconfig
===
--- scsi-misc-2.6.orig/drivers/scsi/Kconfig 2005-08-13 13:53:51.0
+0200
+++ scsi-misc-2.6/drivers/scsi/Kconfig2005-08-15 15:34:56.0
+0200
@@ -229,6 +229,13 @@
each attached iSCSI device to sysfs, say Y.
Otherwise, say N.
+config SCSI_SAS_ATTRS
+ tristate SAS Transport Attributes
+ depends on SCSI
+ help
+ If you wish to export transport-specific information about
+ each attached SAS device to sysfs, say Y.
+
endmenu
menu SCSI low-level drivers
Index: scsi-misc-2.6/drivers/scsi/Makefile
===
--- scsi-misc-2.6.orig/drivers/scsi/Makefile 2005-08-13 13:53:51.0
+0200
+++ scsi-misc-2.6/drivers/scsi/Makefile 2005-08-15 15:35:12.0
+0200
@@ -29,6 +29,7 @@
obj-$(CONFIG_SCSI_SPI_ATTRS) += scsi_transport_spi.o
obj-$(CONFIG_SCSI_FC_ATTRS) += scsi_transport_fc.o
obj-$(CONFIG_SCSI_ISCSI_ATTRS) += scsi_transport_iscsi.o
+obj-$(CONFIG_SCSI_SAS_ATTRS) += scsi_transport_sas.o
obj-$(CONFIG_SCSI_AMIGA7XX) += amiga7xx.o 53c7xx.o
obj-$(CONFIG_A3000_SCSI) += a3000.o wd33c93.o
Index: scsi-misc-2.6/drivers/scsi/scsi_transport_sas.c
===
--- /dev/null 1970-01-01 00:00:00.0 +
+++ scsi-misc-2.6/drivers/scsi/scsi_transport_sas.c 2005-08-15
15:40:00.0 +0200
@@ -0,0 +1,491 @@
+/*
+ * Copyright (C) 2005 Dell Inc.
+ * Released under GPL v2.
+ *
+ * Based on the FC transport class work by James Smart, Emulex Corporation.
+ */
+
+#include linux/init.h
+#include linux/module.h
+#include linux/err.h
+
+#include scsi/scsi_device.h
+#include scsi/scsi_host.h
+#include scsi/scsi_transport.h
+#include scsi/scsi_transport_sas.h
+
+
+#define SAS_TARGET_ATTRS 25
+#define SAS_PORT_ATTRS 25
+
+struct sas_internal {
+ struct scsi_transport_template t;
+ struct sas_function_template *f;
+
+ struct class_device_attribute private_target_attrs[SAS_TARGET_ATTRS];
+ struct class_device_attribute private_port_attrs[SAS_TARGET_ATTRS];
+
+ struct transport_container port_attr_cont;
+
+ /*
+ * The array of null terminated pointers to attributes
+ * needed by scsi_sysfs.c
+ */
+ struct class_device_attribute *target_attrs[SAS_TARGET_ATTRS];
+ struct class_device_attribute *port_attrs[SAS_PORT_ATTRS + 1];
+};
+#define to_sas_internal(tmpl)container_of(tmpl, struct sas_internal,
t)
+
+/*
+ * Hack to allow attributes of the same name in different objects.
+ */
+#define SAS_CLASS_DEVICE_ATTR(_prefix,_name,_mode,_show,_store) \
+ struct class_device_attribute class_device_attr_##_prefix##_##_name = \
+ __ATTR(_name,_mode,_show,_store)
+
+
+/*
+ * Pretty printing helpers
+ */
+
+#define sas_bitfield_name_match(title, table)
Re: [PATCH] minimal SAS transport class
On Mon, Aug 15, 2005 at 11:13:30AM -0400, Luben Tuikov wrote:
+void sas_add_target(struct sas_port *port, struct sas_identify *attached,
+ uint channel, uint target)
+{
+ if (attached-target_port_protocols
+ (SAS_PROTOCOL_SSP|SAS_PROTOCOL_STP|SAS_PROTOCOL_SATA))
+ scsi_scan_target(port-dev, channel, target, ~0, 0, attached);
+}
I've a few questions:
1. What kind of device does the Fusion driver export?
Is this a true end device, or is this the LU in the SSP end device?
I.e. since the Fusion card firmware does everything about SAS there is,
is also LU discovery done in the firmware, or does the firmware export
only the SSP end devices and leave LU discovery to SCSI Core
(as the code suggests)?
It seems to give notification for the actual LU, but doing an LU scan
works if you use the target ID from those reports. Given that I prefer
to do as much as possible in the transport class to have the same
behaviour for different HBAs I'd prefer to not rely on the firmware's
LU scan.
2. Since I saw that (end) devices bind to ports, what is the maximum
number of ports that the Fusion firmware export?
right now it reports the number of physical ports, that's four or eight
for the cards I have. But again I don't have the actual documentation
yet.
3. Will control of SATA and STP devices be given to libata or will
the Fision firmware make those look like SCSI devices?
Fusion makes them look like SCSI devices. As I mentioned I tested this
patch only with SATA disks. But as the command translation for cards
not doing thi in firmware would happen in the -queuecommand path I
don't think the transport class should be involved with it.
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
