Re: [PATCH 2.6.16.18] MSI: Proposed fix for MSI/MSI-X load failure
On Fri, Jun 02, 2006 at 03:21:37PM -0400, Ravinandan Arakali wrote: Symptoms: When a driver is loaded with MSI followed by MSI-X, the load fails indicating that the MSI vector is still active. And vice versa. Suspected rootcause: This happens inspite of driver calling free_irq() followed by pci_disable_msi/pci_disable_msix. This appears to be a kernel bug wherein the pci_disable_msi and pci_disable_msix calls do not clear/unpopulate the msi_desc data structure that was populated by pci_enable_msi/pci_enable_msix. The current MSI code actually does this deliberately, not by accident. It's got a lot of complex code to track devices and vectors and make sure an enable_msi - disable - enable sequence gives a driver the same vector. It also has policies about reserving vectors based on potential hotplug activity etc. Frankly, I've never understood the need for such policies, and am in the process of removing all of them. Proposed fix: Free the MSI vector in pci_disable_msi and all allocated MSI-X vectors in pci_disable_msix. This will break the existing MSI policies. Once you take that away, a whole lot of additional code and complexity can be removed too. That's what I'm working on right now, but such a change is likely too big for -stable. So, I'm ok with this patch if it actually doesn't break MSI/MSI-X. Did you try to repeatedly load/unload an MSI capable driver with this patch? Did you repeatedly try to ifdown/ifup an Ethernet driver that uses MSI? I'm not in a position to test this today, but will try it out next week. thanks, Rajesh - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2.6.16.18] MSI: Proposed fix for MSI/MSI-X load failure
Rajesh, It's possible that the current behavior is by design but once the driver is loaded with MSI, you need a reboot to be able to load MSI-X. And vice versa. I found this rather restrictive. I did test the fix multiple times. For eg. multiple load/unload iterations of MSI followed by multiple load/unload of MSI-X followed by load/unload MSI. That way both transitions(MSI-to-MSI-X and vice versa) are tested. Thanks, Ravi -Original Message- From: Rajesh Shah [mailto:[EMAIL PROTECTED] Sent: Friday, June 02, 2006 2:55 PM To: Ravinandan Arakali Cc: linux-kernel@vger.kernel.org; netdev@vger.kernel.org; Leonid Grossman; Ananda Raju; Sriram Rapuru Subject: Re: [PATCH 2.6.16.18] MSI: Proposed fix for MSI/MSI-X load failure On Fri, Jun 02, 2006 at 03:21:37PM -0400, Ravinandan Arakali wrote: Symptoms: When a driver is loaded with MSI followed by MSI-X, the load fails indicating that the MSI vector is still active. And vice versa. Suspected rootcause: This happens inspite of driver calling free_irq() followed by pci_disable_msi/pci_disable_msix. This appears to be a kernel bug wherein the pci_disable_msi and pci_disable_msix calls do not clear/unpopulate the msi_desc data structure that was populated by pci_enable_msi/pci_enable_msix. The current MSI code actually does this deliberately, not by accident. It's got a lot of complex code to track devices and vectors and make sure an enable_msi - disable - enable sequence gives a driver the same vector. It also has policies about reserving vectors based on potential hotplug activity etc. Frankly, I've never understood the need for such policies, and am in the process of removing all of them. Proposed fix: Free the MSI vector in pci_disable_msi and all allocated MSI-X vectors in pci_disable_msix. This will break the existing MSI policies. Once you take that away, a whole lot of additional code and complexity can be removed too. That's what I'm working on right now, but such a change is likely too big for -stable. So, I'm ok with this patch if it actually doesn't break MSI/MSI-X. Did you try to repeatedly load/unload an MSI capable driver with this patch? Did you repeatedly try to ifdown/ifup an Ethernet driver that uses MSI? I'm not in a position to test this today, but will try it out next week. thanks, Rajesh - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2.6.16.18] MSI: Proposed fix for MSI/MSI-X load failure
Rajesh Shah writes: The current MSI code actually does this deliberately, not by accident. It's got a lot of complex code to track devices and vectors and make sure an enable_msi - disable - enable sequence gives a driver the same vector. It also has policies about reserving vectors based on potential hotplug activity etc. Frankly, I've never understood the need for such policies, and am in the process of removing all of them. Good. We will not be able to support a policy of giving the driver the same vector across an enable_msi/disable/enable sequence on IBM System p machines (64-bit PowerPC), because the firmware controls the MSI allocation, and it doesn't give us the necessary guarantees. Paul. - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
