Re: RFC: Changes for PCI
Richard Henderson wrote: > On Wed, Jun 27, 2001 at 02:07:37PM -0500, Tom Gall wrote: > > Consider also in drivers/pci/pci.c: > > > > The function pci_bus_exists checks based on bus numbers. This function is > > of course used by pci_alloc_primary_bus, which is in turn used by > > pci_scan_bus. As is today, this code can break me the second I'm > > onto my second PCI system domain. > > You'll find that the existing ports that support multiple pci > domains do not number the busses on the secondary domains from > zero. If domain 0 has 3 busses, then domain 1's root bus will > be bus number 3, and so on. Yes I've looked at this in depth, and it does work well to compact things down and conserve on the precious and limited bus numbers. > This approach works quite well in practice, even on machines > with large numbers of pci domains, since there tends to be no > pci-pci busses on domains other than the one containing legacy > i/o widgetry. We have pci-pci bridges in every PCI system domain. Additionally we have cards that have pci-pci bridges on them and certainly they can be plugged in anywhere on the system. Unfortunately the majority of our problem is centered on the fact that in the end we have more than 256 buses. I look forward to this limit disappearing in 2.5. Regards, Tom -- Tom Gall - PPC64 Maintainer "Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom!" (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Richard Henderson wrote: On Wed, Jun 27, 2001 at 02:07:37PM -0500, Tom Gall wrote: Consider also in drivers/pci/pci.c: The function pci_bus_exists checks based on bus numbers. This function is of course used by pci_alloc_primary_bus, which is in turn used by pci_scan_bus. As is today, this code can break me the second I'm onto my second PCI system domain. You'll find that the existing ports that support multiple pci domains do not number the busses on the secondary domains from zero. If domain 0 has 3 busses, then domain 1's root bus will be bus number 3, and so on. Yes I've looked at this in depth, and it does work well to compact things down and conserve on the precious and limited bus numbers. This approach works quite well in practice, even on machines with large numbers of pci domains, since there tends to be no pci-pci busses on domains other than the one containing legacy i/o widgetry. We have pci-pci bridges in every PCI system domain. Additionally we have cards that have pci-pci bridges on them and certainly they can be plugged in anywhere on the system. Unfortunately the majority of our problem is centered on the fact that in the end we have more than 256 buses. I look forward to this limit disappearing in 2.5. Regards, Tom -- Tom Gall - PPC64 Maintainer Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom! (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
"Khachaturov, Vassilii" wrote: > > On Wed, 27 Jun 2001, Jeff Garzik wrote: > > > However, I think the driver (only going by your > description) would be > > more correct to use a pointer to struct pci_dev. We have a > token in the > > kernel that is guaranteed 100% unique to any given PCI device: the > > pointer to its struct pci_dev. > > Is it? With a hotplug device removed and another one added, > isn't there a slight chance that the pci_dev pointer to the new device > will get allocated in place of the old one? If you want to get pedantic, yes ;-) The pci_dev pointer is unique for the lifetime of the PCI device, which works just as well in the example used in the thread. -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: RFC: Changes for PCI
On Wed, 27 Jun 2001, Jeff Garzik wrote: > However, I think the driver (only going by your description) would be > more correct to use a pointer to struct pci_dev. We have a token in the > kernel that is guaranteed 100% unique to any given PCI device: the > pointer to its struct pci_dev. Is it? With a hotplug device removed and another one added, isn't there a slight chance that the pci_dev pointer to the new device will get allocated in place of the old one? Vassilii - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Tom Gall wrote: > Gérard Roudier wrote: > > The driver checks against PCI bus+dev+func in 2 situations: > > > > 1) To apply the boot order that user can set up in the controller NVRAMs. > > 2) To detect buggy double reporting of the same device by the kernel PCI > >code (this made lot of troubles at some time). > > Thanks much for the clarification. Do you still battle buggy double reporting? > Has this been fixed? Is it a bug on some specific architecture? I've seen it occur in 2.2.x in buggy drivers, but in 2.4 the driver should -not- have to check for this. The PCI core takes care of it. -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Gérard Roudier wrote: > > On Wed, 27 Jun 2001, Jeff Garzik wrote: > > > Tom Gall wrote: > > > Well you have device drivers like the symbios scsi driver for instance that > > > tries to determine if it's seen a card before. It does this by looking at the > > > bus,dev etc numbers... It's quite reasonable for two different scsi cards to be > > > on the same bus number, same dev number etc yet they are in different PCI > > > domains. > > > > > > Is this a device driver bug or feature? > > > > I hesitate to call it a device driver bug, because that was likely the > > best decision Gerard could make at the time. > > > > However, I think the driver (only going by your description) would be > > more correct to use a pointer to struct pci_dev. We have a token in the > > kernel that is guaranteed 100% unique to any given PCI device: the > > pointer to its struct pci_dev. > > The driver checks against PCI bus+dev+func in 2 situations: > > 1) To apply the boot order that user can set up in the controller NVRAMs. > 2) To detect buggy double reporting of the same device by the kernel PCI >code (this made lot of troubles at some time). Thanks much for the clarification. Do you still battle buggy double reporting? Has this been fixed? Is it a bug on some specific architecture? > The great bug is to invent useless abstractions that don't match reality. > Such brain masturbation leads to confusion (hence subtle bugs) and > useless software bloatage (thus _real_ resource wastage). Agreed. (A couple of my posts last night didn't make it through... appears that us.ibm.com isn't set up entirely right for ENC) > If we want to handle _real_ PCI bus domains, we just have to add a domain > number to identify a _real_ PCI device. Anything that wants to hide such > reality in some opaque data looks like brain masturbation to me. Again also agreed. Now I'm REALLY anxious for 2.5 8-) > Gérard. Regards, Tom -- Tom Gall - PPC64 Maintainer "Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom!" (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Gérard Roudier wrote: > The driver checks against PCI bus+dev+func in 2 situations: > > 1) To apply the boot order that user can set up in the controller NVRAMs. > 2) To detect buggy double reporting of the same device by the kernel PCI >code (this made lot of troubles at some time). Cool. The premise of the thread was that you merely were checking for uniqueness, not order. That changes our answer... > The great bug is to invent useless abstractions that don't match reality. > Such brain masturbation leads to confusion (hence subtle bugs) and > useless software bloatage (thus _real_ resource wastage). > > If we want to handle _real_ PCI bus domains, we just have to add a domain > number to identify a _real_ PCI device. Anything that wants to hide such > reality in some opaque data looks like brain masturbation to me. I think all of us agree on this: in 2.5, our solution is to have a system domain number, which increases from zero each time you add sbus, pci bus, isa bus, etc. For 2.4, non-x86 arches first had to deal with PCI domains, so the solution was to stick "arch-specific data" into pci_dev->sysdata, which in some cases was the PCI domain number. So, you have an ugly solution in drivers for 2.4 if you need to know PCI domain for what reason, and a clean solution in 2.5. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
On Wed, 27 Jun 2001, Jeff Garzik wrote: > Tom Gall wrote: > > Well you have device drivers like the symbios scsi driver for instance that > > tries to determine if it's seen a card before. It does this by looking at the > > bus,dev etc numbers... It's quite reasonable for two different scsi cards to be > > on the same bus number, same dev number etc yet they are in different PCI > > domains. > > > > Is this a device driver bug or feature? > > I hesitate to call it a device driver bug, because that was likely the > best decision Gerard could make at the time. > > However, I think the driver (only going by your description) would be > more correct to use a pointer to struct pci_dev. We have a token in the > kernel that is guaranteed 100% unique to any given PCI device: the > pointer to its struct pci_dev. The driver checks against PCI bus+dev+func in 2 situations: 1) To apply the boot order that user can set up in the controller NVRAMs. 2) To detect buggy double reporting of the same device by the kernel PCI code (this made lot of troubles at some time). The great bug is to invent useless abstractions that don't match reality. Such brain masturbation leads to confusion (hence subtle bugs) and useless software bloatage (thus _real_ resource wastage). If we want to handle _real_ PCI bus domains, we just have to add a domain number to identify a _real_ PCI device. Anything that wants to hide such reality in some opaque data looks like brain masturbation to me. Gérard. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
On Wed, 27 Jun 2001, Jeff Garzik wrote: Tom Gall wrote: Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... It's quite reasonable for two different scsi cards to be on the same bus number, same dev number etc yet they are in different PCI domains. Is this a device driver bug or feature? I hesitate to call it a device driver bug, because that was likely the best decision Gerard could make at the time. However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. The driver checks against PCI bus+dev+func in 2 situations: 1) To apply the boot order that user can set up in the controller NVRAMs. 2) To detect buggy double reporting of the same device by the kernel PCI code (this made lot of troubles at some time). The great bug is to invent useless abstractions that don't match reality. Such brain masturbation leads to confusion (hence subtle bugs) and useless software bloatage (thus _real_ resource wastage). If we want to handle _real_ PCI bus domains, we just have to add a domain number to identify a _real_ PCI device. Anything that wants to hide such reality in some opaque data looks like brain masturbation to me. Gérard. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Gérard Roudier wrote: The driver checks against PCI bus+dev+func in 2 situations: 1) To apply the boot order that user can set up in the controller NVRAMs. 2) To detect buggy double reporting of the same device by the kernel PCI code (this made lot of troubles at some time). Cool. The premise of the thread was that you merely were checking for uniqueness, not order. That changes our answer... The great bug is to invent useless abstractions that don't match reality. Such brain masturbation leads to confusion (hence subtle bugs) and useless software bloatage (thus _real_ resource wastage). If we want to handle _real_ PCI bus domains, we just have to add a domain number to identify a _real_ PCI device. Anything that wants to hide such reality in some opaque data looks like brain masturbation to me. I think all of us agree on this: in 2.5, our solution is to have a system domain number, which increases from zero each time you add sbus, pci bus, isa bus, etc. For 2.4, non-x86 arches first had to deal with PCI domains, so the solution was to stick arch-specific data into pci_dev-sysdata, which in some cases was the PCI domain number. So, you have an ugly solution in drivers for 2.4 if you need to know PCI domain for what reason, and a clean solution in 2.5. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Gérard Roudier wrote: On Wed, 27 Jun 2001, Jeff Garzik wrote: Tom Gall wrote: Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... It's quite reasonable for two different scsi cards to be on the same bus number, same dev number etc yet they are in different PCI domains. Is this a device driver bug or feature? I hesitate to call it a device driver bug, because that was likely the best decision Gerard could make at the time. However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. The driver checks against PCI bus+dev+func in 2 situations: 1) To apply the boot order that user can set up in the controller NVRAMs. 2) To detect buggy double reporting of the same device by the kernel PCI code (this made lot of troubles at some time). Thanks much for the clarification. Do you still battle buggy double reporting? Has this been fixed? Is it a bug on some specific architecture? The great bug is to invent useless abstractions that don't match reality. Such brain masturbation leads to confusion (hence subtle bugs) and useless software bloatage (thus _real_ resource wastage). Agreed. (A couple of my posts last night didn't make it through... appears that us.ibm.com isn't set up entirely right for ENC) If we want to handle _real_ PCI bus domains, we just have to add a domain number to identify a _real_ PCI device. Anything that wants to hide such reality in some opaque data looks like brain masturbation to me. Again also agreed. Now I'm REALLY anxious for 2.5 8-) Gérard. Regards, Tom -- Tom Gall - PPC64 Maintainer Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom! (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Tom Gall wrote: Gérard Roudier wrote: The driver checks against PCI bus+dev+func in 2 situations: 1) To apply the boot order that user can set up in the controller NVRAMs. 2) To detect buggy double reporting of the same device by the kernel PCI code (this made lot of troubles at some time). Thanks much for the clarification. Do you still battle buggy double reporting? Has this been fixed? Is it a bug on some specific architecture? I've seen it occur in 2.2.x in buggy drivers, but in 2.4 the driver should -not- have to check for this. The PCI core takes care of it. -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: RFC: Changes for PCI
On Wed, 27 Jun 2001, Jeff Garzik wrote: However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. Is it? With a hotplug device removed and another one added, isn't there a slight chance that the pci_dev pointer to the new device will get allocated in place of the old one? Vassilii - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Khachaturov, Vassilii wrote: On Wed, 27 Jun 2001, Jeff Garzik wrote: However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. Is it? With a hotplug device removed and another one added, isn't there a slight chance that the pci_dev pointer to the new device will get allocated in place of the old one? If you want to get pedantic, yes ;-) The pci_dev pointer is unique for the lifetime of the PCI device, which works just as well in the example used in the thread. -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Jeff Garzik wrote: > Tom Gall wrote: > > Well you have device drivers like the symbios scsi driver for instance that > > tries to determine if it's seen a card before. It does this by looking at the > > bus,dev etc numbers... It's quite reasonable for two different scsi cards to be > > on the same bus number, same dev number etc yet they are in different PCI > > domains. > > > > Is this a device driver bug or feature? > > I hesitate to call it a device driver bug, because that was likely the > best decision Gerard could make at the time. I don't doubt it. I wouldn't doubt I've been guilty of simliar things in my past... > However, I think the driver (only going by your description) would be > more correct to use a pointer to struct pci_dev. We have a token in the > kernel that is guaranteed 100% unique to any given PCI device: the > pointer to its struct pci_dev. Sound good. > > > Changing the meaning of dev->bus->number globally seems pointless. If > > > you are going to do that, just do it the right way and introduce another > > > struct member, pci_domain or somesuch. > > Sorry, not pci_domain, just system bus number, for any bus, like we > talked about in the previous discussion. Yes agreed. However do you think it's possible for the additional of such a value now in 2.4.x series? Alan? Linus? Regards, Tom -- Tom Gall - PPC64 Maintainer "Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom!" (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
> Well you have device drivers like the symbios scsi driver for instance that > tries to determine if it's seen a card before. It does this by looking at the > bus,dev etc numbers... Can it be done by comparing struct pci_dev pointers for equal? -- Pete - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Tom Gall wrote: > Well you have device drivers like the symbios scsi driver for instance that > tries to determine if it's seen a card before. It does this by looking at the > bus,dev etc numbers... It's quite reasonable for two different scsi cards to be > on the same bus number, same dev number etc yet they are in different PCI > domains. > > Is this a device driver bug or feature? I hesitate to call it a device driver bug, because that was likely the best decision Gerard could make at the time. However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. > > Changing the meaning of dev->bus->number globally seems pointless. If > > you are going to do that, just do it the right way and introduce another > > struct member, pci_domain or somesuch. > > Right, one could do that and then all the large machine architectures would have > their own implementation for the same problem. That's not necessarily a bad > thing, but some commonality I think would be a good thing. Sorry, not pci_domain, just system bus number, for any bus, like we talked about in the previous discussion. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
> Why not use sysdata like the other arches? > > Changing the meaning of dev->bus->number globally seems pointless. If > you are going to do that, just do it the right way and introduce another > struct member, pci_domain or somesuch. Thats 2.5 material. For 2.4 we should do as davem suggested and make the bus number unique. I do this by just adding 256 to each overlapping host bridge. Anton - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Hi, > The following 3 functions are added. Their purpose is a little > different than to add support for more than 256 buses but they are > important. Skip ahead and I'll explain what they are for > > int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional > hooks provide */ > int (*pci_read_irq)(struct pci_dev *); /* the arch dependant > code a way*/ > int (*pci_fixup_registers)(struct pci_dev *); /* to manage the > registers. */ I do not think these functions are required at all. > The 3 additional functions are hooks so that an architecture has a > chance to make sure things are in order beforehand. pci_read_bases is > for the management and fixup of the BARs. pci_read_irq is the same but > for IRQs. pci_fixup_registers again same idea but for bridge > resources. > > The essense of the design here is to allow you to get in and make sure > everything is ok with the card, bridge etc, beforehand so as to avoid > multiple bus walks. Please look at how other architectures solve this problem. For example on ppc32 we already fix up the BARs if required. There are also hooks to fix the irq, you seem to be duplicating all of this. Anton - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Jeff Garzik wrote: > > Tom Gall wrote: > > The first part changes number, primary, and secondary to unsigned ints from > > chars. What we do is encode the PCI "domain" aka PCI Primary Host Bridge, aka > > pci controller in with the bus number. In our case we do it like this: > > > > pci_controller=dev->bus->number>>8) &0xFF > > bus_number= dev->bus->number&0xFF), > > > > Is this reasonable for everyone? > > Why not use sysdata like the other arches? Hi Jeff, Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... It's quite reasonable for two different scsi cards to be on the same bus number, same dev number etc yet they are in different PCI domains. Is this a device driver bug or feature? > Changing the meaning of dev->bus->number globally seems pointless. If > you are going to do that, just do it the right way and introduce another > struct member, pci_domain or somesuch. Right, one could do that and then all the large machine architectures would have their own implementation for the same problem. That's not necessarily a bad thing, but some commonality I think would be a good thing. > Jeff Regards, Tom -- Tom Gall - PPC64 Maintainer "Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom!" (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Tom Gall wrote: > The first part changes number, primary, and secondary to unsigned ints from > chars. What we do is encode the PCI "domain" aka PCI Primary Host Bridge, aka > pci controller in with the bus number. In our case we do it like this: > > pci_controller=dev->bus->number>>8) &0xFF > bus_number= dev->bus->number&0xFF), > > Is this reasonable for everyone? Why not use sysdata like the other arches? Changing the meaning of dev->bus->number globally seems pointless. If you are going to do that, just do it the right way and introduce another struct member, pci_domain or somesuch. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RFC: Changes for PCI
Hi All,
I'm looking for commentary on the following. As you might recall I had posted
a note a few days back on the lkml about the kinds of systems ppc64 runs on and
the reality of having a boatload of PCI buses out there.
I sure appreciate all the comments and feedback.
Today appended below are two patches that address more or less how we've
solved the problem for ppc64 from an architecture independant standpoint.
Hopefully you will find these patches reasonable or at a minimum a starting part
for more discussion to make this happen. (Our ppc64 work is at linuxppc64.org or
in my dir on kernel.org pub/linux/kernel/people/tgall if you're interested)
Part one is the following changes to include/linux/pci.h
The first part changes number, primary, and secondary to unsigned ints from
chars. What we do is encode the PCI "domain" aka PCI Primary Host Bridge, aka
pci controller in with the bus number. In our case we do it like this:
pci_controller=dev->bus->number>>8) &0xFF
bus_number= dev->bus->number&0xFF),
Is this reasonable for everyone?
subordinate probably doesn't need to go to an int really when I come to think
about it as it can fit in a char, but how about in the future? A switch to an
unsigned int as well seemed prudent.
The following 3 functions are added. Their purpose is a little different than
to add support for more than 256 buses but they are important. Skip ahead and
I'll explain what they are for
int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional
hooks provide */
int (*pci_read_irq)(struct pci_dev *); /* the arch dependant
code a way*/
int (*pci_fixup_registers)(struct pci_dev *); /* to manage the
registers. */
--- linux-2.4.5-ac18/include/linux/pci.hTue Jun 26 21:59:23 2001
+++ linuxppc64_2_4/include/linux/pci.h Wed Jun 27 13:36:26 2001
@@ -60,6 +60,8 @@
#define PCI_CACHE_LINE_SIZE0x0c/* 8 bits */
#define PCI_LATENCY_TIMER 0x0d/* 8 bits */
#define PCI_HEADER_TYPE0x0e/* 8 bits */
+
+#define PCI_MULTIFUNCTION_CARD 0x80 /* Multi-function bit in header. */
#define PCI_HEADER_TYPE_NORMAL0
#define PCI_HEADER_TYPE_BRIDGE 1
#define PCI_HEADER_TYPE_CARDBUS 2
@@ -294,6 +296,13 @@
#define PCIIOC_MMAP_IS_MEM (PCIIOC_BASE | 0x02)/* Set mmap state to MEM
space. */
#define PCIIOC_WRITE_COMBINE (PCIIOC_BASE | 0x03)/* Enable/disable
write-combining. */
+/* Ioctls for /proc/bus/pci/X/Y nodes. */
+#define PCIIOC_BASE('P' << 24 | 'C' << 16 | 'I' << 8)
+#define PCIIOC_CONTROLLER (PCIIOC_BASE | 0x00)/* Get controller for PCI
device. */
+#define PCIIOC_MMAP_IS_IO (PCIIOC_BASE | 0x01)/* Set mmap state to I/O
space. */
+#define PCIIOC_MMAP_IS_MEM (PCIIOC_BASE | 0x02)/* Set mmap state to MEM
space. */
+#define PCIIOC_WRITE_COMBINE (PCIIOC_BASE | 0x03)/* Enable/disable
write-combining. */
+
#ifdef __KERNEL__
#include
@@ -409,10 +418,10 @@
void*sysdata; /* hook for sys-specific extension */
struct proc_dir_entry *procdir; /* directory entry in /proc/bus/pci */
- unsigned char number; /* bus number */
- unsigned char primary;/* number of primary bridge */
- unsigned char secondary; /* number of secondary bridge */
- unsigned char subordinate;/* max number of subordinate buses */
+ unsigned int number; /* pci_controller number + bus number */
+ unsigned int primary; /* number of primary bridge */
+ unsigned int secondary;/* number of secondary bridge */
+ unsigned int subordinate; /* max number of subordinate buses */
charname[48];
unsigned short vendor;
@@ -449,6 +458,9 @@
int (*write_byte)(struct pci_dev *, int where, u8 val);
int (*write_word)(struct pci_dev *, int where, u16 val);
int (*write_dword)(struct pci_dev *, int where, u32 val);
+ int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional
hooks provide */
+ int (*pci_read_irq)(struct pci_dev *); /* the arch
dependant code a way*/
+ int (*)(struct pci_dev *); /* to manage the registers. */
};
struct pbus_set_ranges_data
The other file we changed is drivers/pci/pci.c
The 3 additional functions are hooks so that an architecture has a chance to
make sure things are in order beforehand. pci_read_bases is for the management
and fixup of the BARs. pci_read_irq is the same but for IRQs.
pci_fixup_registers again same idea but for bridge resources.
The essense of the design here is to allow you to get in and make sure
everything is ok with the card, bridge etc, beforehand so as to avoid multiple
bus walks.
diff -u linux-2.4.5-ac18/drivers/pci/pci.c linuxppc64_2_4/drivers/pci/pci.c
--- linux-2.4.5-ac18/drivers/pci/pci.c Tue Jun 26 21:55:54 2001
+++ linuxppc64_2_4/drivers/pci/pci.cTue Jun 26 08:09:31 2001
@@ -1,5 +1,5 @@
/*
- * $Id: pci.c,v 1.91 1999/01/21 13:34:01 davem Exp $
+ *
*
*
RFC: Changes for PCI
Hi All,
I'm looking for commentary on the following. As you might recall I had posted
a note a few days back on the lkml about the kinds of systems ppc64 runs on and
the reality of having a boatload of PCI buses out there.
I sure appreciate all the comments and feedback.
Today appended below are two patches that address more or less how we've
solved the problem for ppc64 from an architecture independant standpoint.
Hopefully you will find these patches reasonable or at a minimum a starting part
for more discussion to make this happen. (Our ppc64 work is at linuxppc64.org or
in my dir on kernel.org pub/linux/kernel/people/tgall if you're interested)
Part one is the following changes to include/linux/pci.h
The first part changes number, primary, and secondary to unsigned ints from
chars. What we do is encode the PCI domain aka PCI Primary Host Bridge, aka
pci controller in with the bus number. In our case we do it like this:
pci_controller=dev-bus-number8) 0xFF
bus_number= dev-bus-number0xFF),
Is this reasonable for everyone?
subordinate probably doesn't need to go to an int really when I come to think
about it as it can fit in a char, but how about in the future? A switch to an
unsigned int as well seemed prudent.
The following 3 functions are added. Their purpose is a little different than
to add support for more than 256 buses but they are important. Skip ahead and
I'll explain what they are for
int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional
hooks provide */
int (*pci_read_irq)(struct pci_dev *); /* the arch dependant
code a way*/
int (*pci_fixup_registers)(struct pci_dev *); /* to manage the
registers. */
--- linux-2.4.5-ac18/include/linux/pci.hTue Jun 26 21:59:23 2001
+++ linuxppc64_2_4/include/linux/pci.h Wed Jun 27 13:36:26 2001
@@ -60,6 +60,8 @@
#define PCI_CACHE_LINE_SIZE0x0c/* 8 bits */
#define PCI_LATENCY_TIMER 0x0d/* 8 bits */
#define PCI_HEADER_TYPE0x0e/* 8 bits */
+
+#define PCI_MULTIFUNCTION_CARD 0x80 /* Multi-function bit in header. */
#define PCI_HEADER_TYPE_NORMAL0
#define PCI_HEADER_TYPE_BRIDGE 1
#define PCI_HEADER_TYPE_CARDBUS 2
@@ -294,6 +296,13 @@
#define PCIIOC_MMAP_IS_MEM (PCIIOC_BASE | 0x02)/* Set mmap state to MEM
space. */
#define PCIIOC_WRITE_COMBINE (PCIIOC_BASE | 0x03)/* Enable/disable
write-combining. */
+/* Ioctls for /proc/bus/pci/X/Y nodes. */
+#define PCIIOC_BASE('P' 24 | 'C' 16 | 'I' 8)
+#define PCIIOC_CONTROLLER (PCIIOC_BASE | 0x00)/* Get controller for PCI
device. */
+#define PCIIOC_MMAP_IS_IO (PCIIOC_BASE | 0x01)/* Set mmap state to I/O
space. */
+#define PCIIOC_MMAP_IS_MEM (PCIIOC_BASE | 0x02)/* Set mmap state to MEM
space. */
+#define PCIIOC_WRITE_COMBINE (PCIIOC_BASE | 0x03)/* Enable/disable
write-combining. */
+
#ifdef __KERNEL__
#include linux/types.h
@@ -409,10 +418,10 @@
void*sysdata; /* hook for sys-specific extension */
struct proc_dir_entry *procdir; /* directory entry in /proc/bus/pci */
- unsigned char number; /* bus number */
- unsigned char primary;/* number of primary bridge */
- unsigned char secondary; /* number of secondary bridge */
- unsigned char subordinate;/* max number of subordinate buses */
+ unsigned int number; /* pci_controller number + bus number */
+ unsigned int primary; /* number of primary bridge */
+ unsigned int secondary;/* number of secondary bridge */
+ unsigned int subordinate; /* max number of subordinate buses */
charname[48];
unsigned short vendor;
@@ -449,6 +458,9 @@
int (*write_byte)(struct pci_dev *, int where, u8 val);
int (*write_word)(struct pci_dev *, int where, u16 val);
int (*write_dword)(struct pci_dev *, int where, u32 val);
+ int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional
hooks provide */
+ int (*pci_read_irq)(struct pci_dev *); /* the arch
dependant code a way*/
+ int (*)(struct pci_dev *); /* to manage the registers. */
};
struct pbus_set_ranges_data
The other file we changed is drivers/pci/pci.c
The 3 additional functions are hooks so that an architecture has a chance to
make sure things are in order beforehand. pci_read_bases is for the management
and fixup of the BARs. pci_read_irq is the same but for IRQs.
pci_fixup_registers again same idea but for bridge resources.
The essense of the design here is to allow you to get in and make sure
everything is ok with the card, bridge etc, beforehand so as to avoid multiple
bus walks.
diff -u linux-2.4.5-ac18/drivers/pci/pci.c linuxppc64_2_4/drivers/pci/pci.c
--- linux-2.4.5-ac18/drivers/pci/pci.c Tue Jun 26 21:55:54 2001
+++ linuxppc64_2_4/drivers/pci/pci.cTue Jun 26 08:09:31 2001
@@ -1,5 +1,5 @@
/*
- * $Id: pci.c,v 1.91 1999/01/21 13:34:01 davem Exp $
+ *
*
* PCI
Re: RFC: Changes for PCI
Tom Gall wrote: The first part changes number, primary, and secondary to unsigned ints from chars. What we do is encode the PCI domain aka PCI Primary Host Bridge, aka pci controller in with the bus number. In our case we do it like this: pci_controller=dev-bus-number8) 0xFF bus_number= dev-bus-number0xFF), Is this reasonable for everyone? Why not use sysdata like the other arches? Changing the meaning of dev-bus-number globally seems pointless. If you are going to do that, just do it the right way and introduce another struct member, pci_domain or somesuch. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Hi, The following 3 functions are added. Their purpose is a little different than to add support for more than 256 buses but they are important. Skip ahead and I'll explain what they are for int (*pci_read_bases)(struct pci_dev *, int cnt,int rom); /* These optional hooks provide */ int (*pci_read_irq)(struct pci_dev *); /* the arch dependant code a way*/ int (*pci_fixup_registers)(struct pci_dev *); /* to manage the registers. */ I do not think these functions are required at all. The 3 additional functions are hooks so that an architecture has a chance to make sure things are in order beforehand. pci_read_bases is for the management and fixup of the BARs. pci_read_irq is the same but for IRQs. pci_fixup_registers again same idea but for bridge resources. The essense of the design here is to allow you to get in and make sure everything is ok with the card, bridge etc, beforehand so as to avoid multiple bus walks. Please look at how other architectures solve this problem. For example on ppc32 we already fix up the BARs if required. There are also hooks to fix the irq, you seem to be duplicating all of this. Anton - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Why not use sysdata like the other arches? Changing the meaning of dev-bus-number globally seems pointless. If you are going to do that, just do it the right way and introduce another struct member, pci_domain or somesuch. Thats 2.5 material. For 2.4 we should do as davem suggested and make the bus number unique. I do this by just adding 256 to each overlapping host bridge. Anton - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Tom Gall wrote: Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... It's quite reasonable for two different scsi cards to be on the same bus number, same dev number etc yet they are in different PCI domains. Is this a device driver bug or feature? I hesitate to call it a device driver bug, because that was likely the best decision Gerard could make at the time. However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. Changing the meaning of dev-bus-number globally seems pointless. If you are going to do that, just do it the right way and introduce another struct member, pci_domain or somesuch. Right, one could do that and then all the large machine architectures would have their own implementation for the same problem. That's not necessarily a bad thing, but some commonality I think would be a good thing. Sorry, not pci_domain, just system bus number, for any bus, like we talked about in the previous discussion. Jeff -- Jeff Garzik | Andre the Giant has a posse. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... Can it be done by comparing struct pci_dev pointers for equal? -- Pete - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RFC: Changes for PCI
Jeff Garzik wrote: Tom Gall wrote: Well you have device drivers like the symbios scsi driver for instance that tries to determine if it's seen a card before. It does this by looking at the bus,dev etc numbers... It's quite reasonable for two different scsi cards to be on the same bus number, same dev number etc yet they are in different PCI domains. Is this a device driver bug or feature? I hesitate to call it a device driver bug, because that was likely the best decision Gerard could make at the time. I don't doubt it. I wouldn't doubt I've been guilty of simliar things in my past... However, I think the driver (only going by your description) would be more correct to use a pointer to struct pci_dev. We have a token in the kernel that is guaranteed 100% unique to any given PCI device: the pointer to its struct pci_dev. Sound good. Changing the meaning of dev-bus-number globally seems pointless. If you are going to do that, just do it the right way and introduce another struct member, pci_domain or somesuch. Sorry, not pci_domain, just system bus number, for any bus, like we talked about in the previous discussion. Yes agreed. However do you think it's possible for the additional of such a value now in 2.4.x series? Alan? Linus? Regards, Tom -- Tom Gall - PPC64 Maintainer Where's the ka-boom? There was Linux Technology Center supposed to be an earth (w) [EMAIL PROTECTED] shattering ka-boom! (w) 507-253-4558 -- Marvin Martian (h) [EMAIL PROTECTED] http://www.ibm.com/linux/ltc/projects/ppc - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
