Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On the other hand, I think I can find a nice lever to move Marvell with, so I'll take this on without needing potentially to compromise your contacts. FWIW Marvell is moving quite nicely... they are actively providing docs under NDA, and sometimes sample code (or even a GPL driver) to active developers. Still trying to push them on opening docs, but one step at a time :) Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 20:14 -0400, Jeff Garzik wrote: > James Bottomley wrote: > > On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: > >> James Bottomley wrote: > >>> On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: > Rather than sitting on this for far too long, I wanted to go ahead and > get this out there. I heard some chips might be trickling out into > public hands. > >>> The first thing to note is about the specs and the pre-production > >>> hardware: the Linux Foundation has mechanism to get both into the hands > >>> of interested developers; if you can point me to contacts, I can at > >>> least get the NDA documentation programme ball rolling. > >> Well, are there interested, motivated, skilled developers with time? :) > >> > >> Otherwise it's a moot point. > > > > We have GregKH's minions ... this would be a good project for them. > > > Same answer ... GregKH has a legion ... lets use it. > > This is a good project for somebody who knows how SAS behaves on the > wire, which is a rather limited group. > > I'm willing to help anyone who proactively moves in this direction, but > they need to be proactive about it... I am the person who will have to > make introductions at Broadcom, and convince Broadcom that new Person is > a highly capable SAS software engineer. > > Telling bcm "I don't know this guy, its doubtful he knows SAS, but we > want to make him the primary engineer anyway" is not a very winning tale :) I wasn't actually planning that ... I was planning to say we have an NDA programme to allow them to get their docs and early access silicon in to the hands of potential developers who can help on the driver. > > SMP and STP, by the way, are simple frame in, frame out. If it > > identifies the initiator and target protocols and allows us to send > > frames, we can probably transmit both protocols. > [...] > > That's going to be a bit of a bit oops ... > [...] > > well, I suppose it was designed for simple direct connection ... > > That's the way it's looking. There are a few avenues for exposing > IDENTIFY and OPEN frames and related details, but no obvious "any frame, > no problem" method like with the Marvell chip. > > IMO it's also indicative that Marvell's chip uses a single set of > command and response queues, whereas Broadcom has command/response > queues for each "port" (bcm's term). Heh, OK ... I'm happy to bet that the market won't be too appreciative of a chip like this, unless its sold as pure SATA. The only real reason for HBAs to speak SAS as well as SATA is that most dual SAS/SATA enclosures have internal expanders, which this chip won't be able to talk to. On the other hand, I think I can find a nice lever to move Marvell with, so I'll take this on without needing potentially to compromise your contacts. James - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 20:14 -0400, Jeff Garzik wrote: James Bottomley wrote: On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. We have GregKH's minions ... this would be a good project for them. Same answer ... GregKH has a legion ... lets use it. This is a good project for somebody who knows how SAS behaves on the wire, which is a rather limited group. I'm willing to help anyone who proactively moves in this direction, but they need to be proactive about it... I am the person who will have to make introductions at Broadcom, and convince Broadcom that new Person is a highly capable SAS software engineer. Telling bcm I don't know this guy, its doubtful he knows SAS, but we want to make him the primary engineer anyway is not a very winning tale :) I wasn't actually planning that ... I was planning to say we have an NDA programme to allow them to get their docs and early access silicon in to the hands of potential developers who can help on the driver. SMP and STP, by the way, are simple frame in, frame out. If it identifies the initiator and target protocols and allows us to send frames, we can probably transmit both protocols. [...] That's going to be a bit of a bit oops ... [...] well, I suppose it was designed for simple direct connection ... That's the way it's looking. There are a few avenues for exposing IDENTIFY and OPEN frames and related details, but no obvious any frame, no problem method like with the Marvell chip. IMO it's also indicative that Marvell's chip uses a single set of command and response queues, whereas Broadcom has command/response queues for each port (bcm's term). Heh, OK ... I'm happy to bet that the market won't be too appreciative of a chip like this, unless its sold as pure SATA. The only real reason for HBAs to speak SAS as well as SATA is that most dual SAS/SATA enclosures have internal expanders, which this chip won't be able to talk to. On the other hand, I think I can find a nice lever to move Marvell with, so I'll take this on without needing potentially to compromise your contacts. James - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On the other hand, I think I can find a nice lever to move Marvell with, so I'll take this on without needing potentially to compromise your contacts. FWIW Marvell is moving quite nicely... they are actively providing docs under NDA, and sometimes sample code (or even a GPL driver) to active developers. Still trying to push them on opening docs, but one step at a time :) Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. We have GregKH's minions ... this would be a good project for them. Same answer ... GregKH has a legion ... lets use it. This is a good project for somebody who knows how SAS behaves on the wire, which is a rather limited group. I'm willing to help anyone who proactively moves in this direction, but they need to be proactive about it... I am the person who will have to make introductions at Broadcom, and convince Broadcom that new Person is a highly capable SAS software engineer. Telling bcm "I don't know this guy, its doubtful he knows SAS, but we want to make him the primary engineer anyway" is not a very winning tale :) SMP and STP, by the way, are simple frame in, frame out. If it identifies the initiator and target protocols and allows us to send frames, we can probably transmit both protocols. [...] That's going to be a bit of a bit oops ... [...] well, I suppose it was designed for simple direct connection ... That's the way it's looking. There are a few avenues for exposing IDENTIFY and OPEN frames and related details, but no obvious "any frame, no problem" method like with the Marvell chip. IMO it's also indicative that Marvell's chip uses a single set of command and response queues, whereas Broadcom has command/response queues for each "port" (bcm's term). Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: > James Bottomley wrote: > > On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: > >> Rather than sitting on this for far too long, I wanted to go ahead and > >> get this out there. I heard some chips might be trickling out into > >> public hands. > > > > The first thing to note is about the specs and the pre-production > > hardware: the Linux Foundation has mechanism to get both into the hands > > of interested developers; if you can point me to contacts, I can at > > least get the NDA documentation programme ball rolling. > > Well, are there interested, motivated, skilled developers with time? :) > > Otherwise it's a moot point. We have GregKH's minions ... this would be a good project for them. > >> This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use > >> the vaunted libsas. Notes: > > > > I wouldn't call it "vaunted" but it's been a fun project. > > That was sarcasm :) > > libsas is a big 'ole pain, that I'm finding has many aic94xx-isms buried > in the lib. Well, that's hardly surprising ... it was written by Adaptec and there was no other device to help with its production. > >> * A quick glance at the FIXMEs will tell you obviously doesn't work. > > > > The first thing I really noted is that SMP and STP protocol support is > > stubbed out ... you really can't do anything other than direct device > > connection without them. > > That's sorta the way I read the hardware docs, too. > > I have some engineering questions pending with Broadcom, but from my > read, SMP and STP don't seem supported. That would effectively render the device pretty much useless. The maing benefit of SAS is that you can support expanders, which are the predominant connection infrastructure. Direct connect SAS is OK, but SATA tends to be cheaper. > >> * The hardware is quite simple and straightforward and easy to program > >> in an efficient way: each SAS port has a command queue (DMA ring) and > >> a response queue (DMA ring). Or if in SATA mode, just a command > >> queue. > > > > That's not such a bad way of doing it ... it pretty much mimics the wire > > protocol, which is simply frame in/frame out for SAS. > > Yep. The hardware (on my end of the spectrum) seems to be moving > towards forcing software to generate all "packets," except (a) data > frames [generated via DMA engine] or (b) special frames that need to > modify the software-generate frame. SMP and STP, by the way, are simple frame in, frame out. If it identifies the initiator and target protocols and allows us to send frames, we can probably transmit both protocols. > >> * The SAS/SATA negotiation is largely out of our hands. The silicon > >> does its thing, and then tells us what type of device connected. We > >> are then expected to switch the port to either SAS mode or SATA mode, > >> accordingly. > >> > >> * There is no firmware or anything. Just DMA and register bitbanging. > >> We have plenty of low-level control. > >> > >> * The state of SAS/SATA integration is perpetually pathetic. Updates > >> in this area are likely. There's a rumor Brian King @ IBM may look > >> into this area too. > >> > >> * This driver pretty much completely lacks exception handling. > > > > I also note there's a slight nomenclature issue which will trip up SAS > > people. All through the driver, you seem to use the word "port" to > > refer to a physical phy. the struct bs_port seems to actually be a phy > > descriptor ... unless there's some missing phy<->port setup logic that > > will be in the final driver? The trouble is that phys and ports are > > distinct (and not equivalent) objects in SAS. > > Nomemclature came straight from the hardware docs, I'm afraid. > > Comparing with the Marvell hardware, I can see how (with Marvell) wide > ports can be set up, and the port/phy distinction is easily programmable > depending on the situation. > > Not so with BCM8603. That's going to be a bit of a bit oops ... > The only places where the docs mention SMP and STP at all is in the SAS > outgoing DMA descriptor docs, when you fill in connection type. The > _only_ other mention of SMP or STP at all is a note saying neither is > supported. So, even the docs contradict themselves, but overall I have > the feeling that SMP/STP are out of my hands. Heh, well, I suppose it was designed for simple direct connection ... > I wonder if Broadcom's interface is born out of the closed RAID-on-chip > product that this is descended from. > > Hopefully more knowledge will be gained soon, as I debug simple SAS and > SATA device plug/unplug, and ask Broadcom questions. > > > >> As an aside, I am also writing a driver for Marvell chips that behave > >> quite similarly to this chip. It seems the future of storage might look > >> like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the > >> volume marketspace at least. > > > > If you have a
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Jeff Garzik wrote: Just for everybody's information, the Marvell SAS/SATA chip for which I'm also writing a driver definitely supports all of that: SMP, STP, wire ports, SCSI target mode, even SATA target mode. s/wire/wide/ of course - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Douglas Gilbert wrote: Is the lack of SMP support a driver limitation or is it the silicon? How about support for wide ports (i.e. when 2 or more HBA phys are attached to remote phys which have the same SAS addresses)? Last question: can the chip run in SCSI target mode? Just for everybody's information, the Marvell SAS/SATA chip for which I'm also writing a driver definitely supports all of that: SMP, STP, wire ports, SCSI target mode, even SATA target mode. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use the vaunted libsas. Notes: I wouldn't call it "vaunted" but it's been a fun project. That was sarcasm :) libsas is a big 'ole pain, that I'm finding has many aic94xx-isms buried in the lib. * A quick glance at the FIXMEs will tell you obviously doesn't work. The first thing I really noted is that SMP and STP protocol support is stubbed out ... you really can't do anything other than direct device connection without them. That's sorta the way I read the hardware docs, too. I have some engineering questions pending with Broadcom, but from my read, SMP and STP don't seem supported. * The hardware is quite simple and straightforward and easy to program in an efficient way: each SAS port has a command queue (DMA ring) and a response queue (DMA ring). Or if in SATA mode, just a command queue. That's not such a bad way of doing it ... it pretty much mimics the wire protocol, which is simply frame in/frame out for SAS. Yep. The hardware (on my end of the spectrum) seems to be moving towards forcing software to generate all "packets," except (a) data frames [generated via DMA engine] or (b) special frames that need to modify the software-generate frame. * The SAS/SATA negotiation is largely out of our hands. The silicon does its thing, and then tells us what type of device connected. We are then expected to switch the port to either SAS mode or SATA mode, accordingly. * There is no firmware or anything. Just DMA and register bitbanging. We have plenty of low-level control. * The state of SAS/SATA integration is perpetually pathetic. Updates in this area are likely. There's a rumor Brian King @ IBM may look into this area too. * This driver pretty much completely lacks exception handling. I also note there's a slight nomenclature issue which will trip up SAS people. All through the driver, you seem to use the word "port" to refer to a physical phy. the struct bs_port seems to actually be a phy descriptor ... unless there's some missing phy<->port setup logic that will be in the final driver? The trouble is that phys and ports are distinct (and not equivalent) objects in SAS. Nomemclature came straight from the hardware docs, I'm afraid. Comparing with the Marvell hardware, I can see how (with Marvell) wide ports can be set up, and the port/phy distinction is easily programmable depending on the situation. Not so with BCM8603. The only places where the docs mention SMP and STP at all is in the SAS outgoing DMA descriptor docs, when you fill in connection type. The _only_ other mention of SMP or STP at all is a note saying neither is supported. So, even the docs contradict themselves, but overall I have the feeling that SMP/STP are out of my hands. I wonder if Broadcom's interface is born out of the closed RAID-on-chip product that this is descended from. Hopefully more knowledge will be gained soon, as I debug simple SAS and SATA device plug/unplug, and ask Broadcom questions. As an aside, I am also writing a driver for Marvell chips that behave quite similarly to this chip. It seems the future of storage might look like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the volume marketspace at least. If you have a contact here too, I can get the LF NDA and hardware programmes rolling. Same response as at the top :) Marvell is actually better at responding than Broadcom, but I'm quite reluctant to make /another/ introduction (already did so for one hacker) that leads nowhere. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Douglas Gilbert wrote: Is the lack of SMP support a driver limitation or is it the silicon? Open question (pending w/ BCM). It looks like the answer is "silicon limitation". How about support for wide ports Open question (w/ BCM). It looks like the answer is "no support." Last question: can the chip run in SCSI target mode? AFAICT, no. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: > Rather than sitting on this for far too long, I wanted to go ahead and > get this out there. I heard some chips might be trickling out into > public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. > This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use > the vaunted libsas. Notes: I wouldn't call it "vaunted" but it's been a fun project. > * A quick glance at the FIXMEs will tell you obviously doesn't work. The first thing I really noted is that SMP and STP protocol support is stubbed out ... you really can't do anything other than direct device connection without them. > * The hardware is quite simple and straightforward and easy to program > in an efficient way: each SAS port has a command queue (DMA ring) and > a response queue (DMA ring). Or if in SATA mode, just a command > queue. That's not such a bad way of doing it ... it pretty much mimics the wire protocol, which is simply frame in/frame out for SAS. > * The SAS/SATA negotiation is largely out of our hands. The silicon > does its thing, and then tells us what type of device connected. We > are then expected to switch the port to either SAS mode or SATA mode, > accordingly. > > * There is no firmware or anything. Just DMA and register bitbanging. > We have plenty of low-level control. > > * The state of SAS/SATA integration is perpetually pathetic. Updates > in this area are likely. There's a rumor Brian King @ IBM may look > into this area too. > > * This driver pretty much completely lacks exception handling. I also note there's a slight nomenclature issue which will trip up SAS people. All through the driver, you seem to use the word "port" to refer to a physical phy. the struct bs_port seems to actually be a phy descriptor ... unless there's some missing phy<->port setup logic that will be in the final driver? The trouble is that phys and ports are distinct (and not equivalent) objects in SAS. > As an aside, I am also writing a driver for Marvell chips that behave > quite similarly to this chip. It seems the future of storage might look > like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the > volume marketspace at least. If you have a contact here too, I can get the LF NDA and hardware programmes rolling. James - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Jeff Garzik wrote: > Rather than sitting on this for far too long, I wanted to go ahead and > get this out there. I heard some chips might be trickling out into > public hands. > > This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use > the vaunted libsas. Notes: > > * A quick glance at the FIXMEs will tell you obviously doesn't work. > > * The hardware is quite simple and straightforward and easy to program > in an efficient way: each SAS port has a command queue (DMA ring) and > a response queue (DMA ring). Or if in SATA mode, just a command > queue. > > * The SAS/SATA negotiation is largely out of our hands. The silicon > does its thing, and then tells us what type of device connected. We > are then expected to switch the port to either SAS mode or SATA mode, > accordingly. > > * There is no firmware or anything. Just DMA and register bitbanging. > We have plenty of low-level control. > > * The state of SAS/SATA integration is perpetually pathetic. Updates > in this area are likely. There's a rumor Brian King @ IBM may look > into this area too. > > * This driver pretty much completely lacks exception handling. > > > As an aside, I am also writing a driver for Marvell chips that behave > quite similarly to this chip. It seems the future of storage might look > like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the > volume marketspace at least. Jeff, Is the lack of SMP support a driver limitation or is it the silicon? How about support for wide ports (i.e. when 2 or more HBA phys are attached to remote phys which have the same SAS addresses)? Last question: can the chip run in SCSI target mode? Doug Gilbert - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use the vaunted libsas. Notes: * A quick glance at the FIXMEs will tell you obviously doesn't work. * The hardware is quite simple and straightforward and easy to program in an efficient way: each SAS port has a command queue (DMA ring) and a response queue (DMA ring). Or if in SATA mode, just a command queue. * The SAS/SATA negotiation is largely out of our hands. The silicon does its thing, and then tells us what type of device connected. We are then expected to switch the port to either SAS mode or SATA mode, accordingly. * There is no firmware or anything. Just DMA and register bitbanging. We have plenty of low-level control. * The state of SAS/SATA integration is perpetually pathetic. Updates in this area are likely. There's a rumor Brian King @ IBM may look into this area too. * This driver pretty much completely lacks exception handling. As an aside, I am also writing a driver for Marvell chips that behave quite similarly to this chip. It seems the future of storage might look like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the volume marketspace at least. Jeff, Is the lack of SMP support a driver limitation or is it the silicon? How about support for wide ports (i.e. when 2 or more HBA phys are attached to remote phys which have the same SAS addresses)? Last question: can the chip run in SCSI target mode? Doug Gilbert - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use the vaunted libsas. Notes: I wouldn't call it vaunted but it's been a fun project. * A quick glance at the FIXMEs will tell you obviously doesn't work. The first thing I really noted is that SMP and STP protocol support is stubbed out ... you really can't do anything other than direct device connection without them. * The hardware is quite simple and straightforward and easy to program in an efficient way: each SAS port has a command queue (DMA ring) and a response queue (DMA ring). Or if in SATA mode, just a command queue. That's not such a bad way of doing it ... it pretty much mimics the wire protocol, which is simply frame in/frame out for SAS. * The SAS/SATA negotiation is largely out of our hands. The silicon does its thing, and then tells us what type of device connected. We are then expected to switch the port to either SAS mode or SATA mode, accordingly. * There is no firmware or anything. Just DMA and register bitbanging. We have plenty of low-level control. * The state of SAS/SATA integration is perpetually pathetic. Updates in this area are likely. There's a rumor Brian King @ IBM may look into this area too. * This driver pretty much completely lacks exception handling. I also note there's a slight nomenclature issue which will trip up SAS people. All through the driver, you seem to use the word port to refer to a physical phy. the struct bs_port seems to actually be a phy descriptor ... unless there's some missing phy-port setup logic that will be in the final driver? The trouble is that phys and ports are distinct (and not equivalent) objects in SAS. As an aside, I am also writing a driver for Marvell chips that behave quite similarly to this chip. It seems the future of storage might look like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the volume marketspace at least. If you have a contact here too, I can get the LF NDA and hardware programmes rolling. James - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Douglas Gilbert wrote: Is the lack of SMP support a driver limitation or is it the silicon? Open question (pending w/ BCM). It looks like the answer is silicon limitation. How about support for wide ports Open question (w/ BCM). It looks like the answer is no support. Last question: can the chip run in SCSI target mode? AFAICT, no. Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use the vaunted libsas. Notes: I wouldn't call it vaunted but it's been a fun project. That was sarcasm :) libsas is a big 'ole pain, that I'm finding has many aic94xx-isms buried in the lib. * A quick glance at the FIXMEs will tell you obviously doesn't work. The first thing I really noted is that SMP and STP protocol support is stubbed out ... you really can't do anything other than direct device connection without them. That's sorta the way I read the hardware docs, too. I have some engineering questions pending with Broadcom, but from my read, SMP and STP don't seem supported. * The hardware is quite simple and straightforward and easy to program in an efficient way: each SAS port has a command queue (DMA ring) and a response queue (DMA ring). Or if in SATA mode, just a command queue. That's not such a bad way of doing it ... it pretty much mimics the wire protocol, which is simply frame in/frame out for SAS. Yep. The hardware (on my end of the spectrum) seems to be moving towards forcing software to generate all packets, except (a) data frames [generated via DMA engine] or (b) special frames that need to modify the software-generate frame. * The SAS/SATA negotiation is largely out of our hands. The silicon does its thing, and then tells us what type of device connected. We are then expected to switch the port to either SAS mode or SATA mode, accordingly. * There is no firmware or anything. Just DMA and register bitbanging. We have plenty of low-level control. * The state of SAS/SATA integration is perpetually pathetic. Updates in this area are likely. There's a rumor Brian King @ IBM may look into this area too. * This driver pretty much completely lacks exception handling. I also note there's a slight nomenclature issue which will trip up SAS people. All through the driver, you seem to use the word port to refer to a physical phy. the struct bs_port seems to actually be a phy descriptor ... unless there's some missing phy-port setup logic that will be in the final driver? The trouble is that phys and ports are distinct (and not equivalent) objects in SAS. Nomemclature came straight from the hardware docs, I'm afraid. Comparing with the Marvell hardware, I can see how (with Marvell) wide ports can be set up, and the port/phy distinction is easily programmable depending on the situation. Not so with BCM8603. The only places where the docs mention SMP and STP at all is in the SAS outgoing DMA descriptor docs, when you fill in connection type. The _only_ other mention of SMP or STP at all is a note saying neither is supported. So, even the docs contradict themselves, but overall I have the feeling that SMP/STP are out of my hands. I wonder if Broadcom's interface is born out of the closed RAID-on-chip product that this is descended from. Hopefully more knowledge will be gained soon, as I debug simple SAS and SATA device plug/unplug, and ask Broadcom questions. As an aside, I am also writing a driver for Marvell chips that behave quite similarly to this chip. It seems the future of storage might look like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the volume marketspace at least. If you have a contact here too, I can get the LF NDA and hardware programmes rolling. Same response as at the top :) Marvell is actually better at responding than Broadcom, but I'm quite reluctant to make /another/ introduction (already did so for one hacker) that leads nowhere. Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Douglas Gilbert wrote: Is the lack of SMP support a driver limitation or is it the silicon? How about support for wide ports (i.e. when 2 or more HBA phys are attached to remote phys which have the same SAS addresses)? Last question: can the chip run in SCSI target mode? Just for everybody's information, the Marvell SAS/SATA chip for which I'm also writing a driver definitely supports all of that: SMP, STP, wire ports, SCSI target mode, even SATA target mode. Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Jeff Garzik wrote: Just for everybody's information, the Marvell SAS/SATA chip for which I'm also writing a driver definitely supports all of that: SMP, STP, wire ports, SCSI target mode, even SATA target mode. s/wire/wide/ of course - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. We have GregKH's minions ... this would be a good project for them. This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use the vaunted libsas. Notes: I wouldn't call it vaunted but it's been a fun project. That was sarcasm :) libsas is a big 'ole pain, that I'm finding has many aic94xx-isms buried in the lib. Well, that's hardly surprising ... it was written by Adaptec and there was no other device to help with its production. * A quick glance at the FIXMEs will tell you obviously doesn't work. The first thing I really noted is that SMP and STP protocol support is stubbed out ... you really can't do anything other than direct device connection without them. That's sorta the way I read the hardware docs, too. I have some engineering questions pending with Broadcom, but from my read, SMP and STP don't seem supported. That would effectively render the device pretty much useless. The maing benefit of SAS is that you can support expanders, which are the predominant connection infrastructure. Direct connect SAS is OK, but SATA tends to be cheaper. * The hardware is quite simple and straightforward and easy to program in an efficient way: each SAS port has a command queue (DMA ring) and a response queue (DMA ring). Or if in SATA mode, just a command queue. That's not such a bad way of doing it ... it pretty much mimics the wire protocol, which is simply frame in/frame out for SAS. Yep. The hardware (on my end of the spectrum) seems to be moving towards forcing software to generate all packets, except (a) data frames [generated via DMA engine] or (b) special frames that need to modify the software-generate frame. SMP and STP, by the way, are simple frame in, frame out. If it identifies the initiator and target protocols and allows us to send frames, we can probably transmit both protocols. * The SAS/SATA negotiation is largely out of our hands. The silicon does its thing, and then tells us what type of device connected. We are then expected to switch the port to either SAS mode or SATA mode, accordingly. * There is no firmware or anything. Just DMA and register bitbanging. We have plenty of low-level control. * The state of SAS/SATA integration is perpetually pathetic. Updates in this area are likely. There's a rumor Brian King @ IBM may look into this area too. * This driver pretty much completely lacks exception handling. I also note there's a slight nomenclature issue which will trip up SAS people. All through the driver, you seem to use the word port to refer to a physical phy. the struct bs_port seems to actually be a phy descriptor ... unless there's some missing phy-port setup logic that will be in the final driver? The trouble is that phys and ports are distinct (and not equivalent) objects in SAS. Nomemclature came straight from the hardware docs, I'm afraid. Comparing with the Marvell hardware, I can see how (with Marvell) wide ports can be set up, and the port/phy distinction is easily programmable depending on the situation. Not so with BCM8603. That's going to be a bit of a bit oops ... The only places where the docs mention SMP and STP at all is in the SAS outgoing DMA descriptor docs, when you fill in connection type. The _only_ other mention of SMP or STP at all is a note saying neither is supported. So, even the docs contradict themselves, but overall I have the feeling that SMP/STP are out of my hands. Heh, well, I suppose it was designed for simple direct connection ... I wonder if Broadcom's interface is born out of the closed RAID-on-chip product that this is descended from. Hopefully more knowledge will be gained soon, as I debug simple SAS and SATA device plug/unplug, and ask Broadcom questions. As an aside, I am also writing a driver for Marvell chips that behave quite similarly to this chip. It seems the future of storage might look like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the volume marketspace at least. If you have a contact here too, I can get the LF NDA and hardware programmes rolling. Same response as at the top :) Marvell is actually better at responding than Broadcom, but I'm quite
Re: [PATCH] Broadcom 8603 SAS/SATA driver, rough draft
James Bottomley wrote: On Sun, 2007-09-23 at 19:43 -0400, Jeff Garzik wrote: James Bottomley wrote: On Sun, 2007-09-23 at 00:04 -0400, Jeff Garzik wrote: Rather than sitting on this for far too long, I wanted to go ahead and get this out there. I heard some chips might be trickling out into public hands. The first thing to note is about the specs and the pre-production hardware: the Linux Foundation has mechanism to get both into the hands of interested developers; if you can point me to contacts, I can at least get the NDA documentation programme ball rolling. Well, are there interested, motivated, skilled developers with time? :) Otherwise it's a moot point. We have GregKH's minions ... this would be a good project for them. Same answer ... GregKH has a legion ... lets use it. This is a good project for somebody who knows how SAS behaves on the wire, which is a rather limited group. I'm willing to help anyone who proactively moves in this direction, but they need to be proactive about it... I am the person who will have to make introductions at Broadcom, and convince Broadcom that new Person is a highly capable SAS software engineer. Telling bcm I don't know this guy, its doubtful he knows SAS, but we want to make him the primary engineer anyway is not a very winning tale :) SMP and STP, by the way, are simple frame in, frame out. If it identifies the initiator and target protocols and allows us to send frames, we can probably transmit both protocols. [...] That's going to be a bit of a bit oops ... [...] well, I suppose it was designed for simple direct connection ... That's the way it's looking. There are a few avenues for exposing IDENTIFY and OPEN frames and related details, but no obvious any frame, no problem method like with the Marvell chip. IMO it's also indicative that Marvell's chip uses a single set of command and response queues, whereas Broadcom has command/response queues for each port (bcm's term). Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Rather than sitting on this for far too long, I wanted to go ahead and
get this out there. I heard some chips might be trickling out into
public hands.
This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use
the vaunted libsas. Notes:
* A quick glance at the FIXMEs will tell you obviously doesn't work.
* The hardware is quite simple and straightforward and easy to program
in an efficient way: each SAS port has a command queue (DMA ring) and
a response queue (DMA ring). Or if in SATA mode, just a command
queue.
* The SAS/SATA negotiation is largely out of our hands. The silicon
does its thing, and then tells us what type of device connected. We
are then expected to switch the port to either SAS mode or SATA mode,
accordingly.
* There is no firmware or anything. Just DMA and register bitbanging.
We have plenty of low-level control.
* The state of SAS/SATA integration is perpetually pathetic. Updates
in this area are likely. There's a rumor Brian King @ IBM may look
into this area too.
* This driver pretty much completely lacks exception handling.
As an aside, I am also writing a driver for Marvell chips that behave
quite similarly to this chip. It seems the future of storage might look
like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the
volume marketspace at least.
The 'broadsas' branch of
master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git broadsas
contains the following updates:
drivers/scsi/Kconfig| 10 +
drivers/scsi/Makefile |1 +
drivers/scsi/broadsas.c | 997 +++
3 files changed, 1008 insertions(+), 0 deletions(-)
create mode 100644 drivers/scsi/broadsas.c
Jeff Garzik (1):
Add rough draft Broadcom 8603 SAS/SATA driver.
diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index 6f2c71e..44fa3a9 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -486,6 +486,16 @@ config SCSI_AIC7XXX_OLD
source "drivers/scsi/aic7xxx/Kconfig.aic79xx"
source "drivers/scsi/aic94xx/Kconfig"
+config SCSI_BROADSAS
+ tristate "Broadcom 8603 SAS/SATA support"
+ depends on PCI
+ select SCSI_SAS_LIBSAS
+ help
+ This driver supports Broadcom SAS/SATA PCI devices.
+
+ To compile this driver as a module, choose M here: the
+ module will be called broadsas.
+
# All the I2O code and drivers do not seem to be 64bit safe.
config SCSI_DPT_I2O
tristate "Adaptec I2O RAID support "
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index 86a7ba7..8f052c9 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -72,6 +72,7 @@ obj-$(CONFIG_SCSI_AIC79XX)+= aic7xxx/
obj-$(CONFIG_SCSI_AACRAID) += aacraid/
obj-$(CONFIG_SCSI_AIC7XXX_OLD) += aic7xxx_old.o
obj-$(CONFIG_SCSI_AIC94XX) += aic94xx/
+obj-$(CONFIG_SCSI_BROADSAS)+= broadsas.o
obj-$(CONFIG_SCSI_IPS) += ips.o
obj-$(CONFIG_SCSI_FD_MCS) += fd_mcs.o
obj-$(CONFIG_SCSI_FUTURE_DOMAIN)+= fdomain.o
diff --git a/drivers/scsi/broadsas.c b/drivers/scsi/broadsas.c
new file mode 100644
index 000..a4276ec
--- /dev/null
+++ b/drivers/scsi/broadsas.c
@@ -0,0 +1,997 @@
+/*
+ * broadsas.c - Broadcom 8603 SAS/SATA support
+ *
+ * Copyright 2007 Red Hat, Inc.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define DRV_NAME "broadsas"
+#define DRV_VERSION "0.1"
+
+struct bs_info;
+struct bs_port;
+
+#define br32(reg) readl(regs + BS_##reg)
+#define bw32(reg,val) writel((val), regs + BS_##reg)
+#define bw32_f(reg,val)do {\
+ writel((val), regs + BS_##reg); \
+ readl(regs + BS_##reg); \
+ } while (0)
+
+/* driver compile-time configuration */
+enum driver_configuration {
+ BS_CMDQ_SZ = 128, /* OK: 2, 4, 8, 16, 32, 64, 128, 256 */
+ BS_RSPQ_SZ = 128, /* OK: 2, 4, 8, 16, 32, 64, 128, 256 */
+ BS_MAX_PRD = 256, /* S/G per slot; my arbitrary guess */
+ BS_RSP_BUF_SZ = 512, /* Response buffer size; OK: 0-64k */
+};
+
+/* unchangeable hardware details */
+enum hardware_details {
+ BS_PORTS= 8,/* number
[PATCH] Broadcom 8603 SAS/SATA driver, rough draft
Rather than sitting on this for far too long, I wanted to go ahead and
get this out there. I heard some chips might be trickling out into
public hands.
This is a bare bones Broadcom 8603 SAS+SATA driver, attempting to use
the vaunted libsas. Notes:
* A quick glance at the FIXMEs will tell you obviously doesn't work.
* The hardware is quite simple and straightforward and easy to program
in an efficient way: each SAS port has a command queue (DMA ring) and
a response queue (DMA ring). Or if in SATA mode, just a command
queue.
* The SAS/SATA negotiation is largely out of our hands. The silicon
does its thing, and then tells us what type of device connected. We
are then expected to switch the port to either SAS mode or SATA mode,
accordingly.
* There is no firmware or anything. Just DMA and register bitbanging.
We have plenty of low-level control.
* The state of SAS/SATA integration is perpetually pathetic. Updates
in this area are likely. There's a rumor Brian King @ IBM may look
into this area too.
* This driver pretty much completely lacks exception handling.
As an aside, I am also writing a driver for Marvell chips that behave
quite similarly to this chip. It seems the future of storage might look
like these Broadcom and Marvell SAS+SATA DMA ring interfaces, in the
volume marketspace at least.
The 'broadsas' branch of
master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git broadsas
contains the following updates:
drivers/scsi/Kconfig| 10 +
drivers/scsi/Makefile |1 +
drivers/scsi/broadsas.c | 997 +++
3 files changed, 1008 insertions(+), 0 deletions(-)
create mode 100644 drivers/scsi/broadsas.c
Jeff Garzik (1):
Add rough draft Broadcom 8603 SAS/SATA driver.
diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index 6f2c71e..44fa3a9 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -486,6 +486,16 @@ config SCSI_AIC7XXX_OLD
source drivers/scsi/aic7xxx/Kconfig.aic79xx
source drivers/scsi/aic94xx/Kconfig
+config SCSI_BROADSAS
+ tristate Broadcom 8603 SAS/SATA support
+ depends on PCI
+ select SCSI_SAS_LIBSAS
+ help
+ This driver supports Broadcom SAS/SATA PCI devices.
+
+ To compile this driver as a module, choose M here: the
+ module will be called broadsas.
+
# All the I2O code and drivers do not seem to be 64bit safe.
config SCSI_DPT_I2O
tristate Adaptec I2O RAID support
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index 86a7ba7..8f052c9 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -72,6 +72,7 @@ obj-$(CONFIG_SCSI_AIC79XX)+= aic7xxx/
obj-$(CONFIG_SCSI_AACRAID) += aacraid/
obj-$(CONFIG_SCSI_AIC7XXX_OLD) += aic7xxx_old.o
obj-$(CONFIG_SCSI_AIC94XX) += aic94xx/
+obj-$(CONFIG_SCSI_BROADSAS)+= broadsas.o
obj-$(CONFIG_SCSI_IPS) += ips.o
obj-$(CONFIG_SCSI_FD_MCS) += fd_mcs.o
obj-$(CONFIG_SCSI_FUTURE_DOMAIN)+= fdomain.o
diff --git a/drivers/scsi/broadsas.c b/drivers/scsi/broadsas.c
new file mode 100644
index 000..a4276ec
--- /dev/null
+++ b/drivers/scsi/broadsas.c
@@ -0,0 +1,997 @@
+/*
+ * broadsas.c - Broadcom 8603 SAS/SATA support
+ *
+ * Copyright 2007 Red Hat, Inc.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+#include linux/kernel.h
+#include linux/module.h
+#include linux/pci.h
+#include linux/interrupt.h
+#include linux/spinlock.h
+#include linux/delay.h
+#include linux/dma-mapping.h
+#include scsi/libsas.h
+#include asm/io.h
+
+#define DRV_NAME broadsas
+#define DRV_VERSION 0.1
+
+struct bs_info;
+struct bs_port;
+
+#define br32(reg) readl(regs + BS_##reg)
+#define bw32(reg,val) writel((val), regs + BS_##reg)
+#define bw32_f(reg,val)do {\
+ writel((val), regs + BS_##reg); \
+ readl(regs + BS_##reg); \
+ } while (0)
+
+/* driver compile-time configuration */
+enum driver_configuration {
+ BS_CMDQ_SZ = 128, /* OK: 2, 4, 8, 16, 32, 64, 128, 256 */
+ BS_RSPQ_SZ = 128, /* OK: 2, 4, 8, 16, 32, 64, 128, 256 */
+ BS_MAX_PRD = 256, /* S/G per slot; my arbitrary guess */
+ BS_RSP_BUF_SZ = 512, /* Response buffer size; OK: 0-64k */
+};
