Thanks Lucy for the detailed explanation. The problem I have is for 4 separate devices (pci functions). Only the first instance is able to allocate MSI-X vectors and the other once fail to a level interrupt (x86 on b111). My understanding that even w/o IRM the limitation is per device rather than system wide. Have I missed anything? Eitan
-----Original Message----- From: xiuyan.w...@sun.com [mailto:xiuyan.w...@sun.com] Sent: Monday, September 14, 2009 8:23 PM To: Eitan Eliahu Cc: driver-discuss@opensolaris.org Subject: Re: [driver-discuss] MSI-X vector shortage Hi Eitan, Generally, MSI/X interrupts are limited by the root complex implementation. So the OS needs some relatively fair allocation algorithm to assign such limited resources. In Solaris, without IRM, the default maximum number of MSI/X interrupt can be allocated to a pci function is 8 for SPARC and the number is 2 for x86. Tt can be tuned by increasing ddi_msix_alloc_limit through /etc/system. But on x86, the total number of interrupt vectors for interrupt priority 5 is 32 (5 is the default interrupt priority for network devices), so you can not allocate more interrupts than 32 even with tuning. With IRM, the above limitation is removed and the allocation algorithm is more flexible. But for x86, the driver can only allocate more interrupts for a pci function when the fix of the below CR is available: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6669984 In my opinion, it's quite challenging for a IRM-capable driver to handle the situation of interrupt reclaim when the system is short of interrupt resources. Having said the above, ddi_intr_dup_handler() might be able to address your issue if you do want to use large number of interrupts, which lets more MSI-X "slot" to share the same interrupt vector if the driver can not get as more interrupts as desired. But it only works on SPARC. Please see the man page for the details on the usage. Thanks, Lucy On 2009/09/15 06:26, Eitan Eliahu wrote: > It seems that the driver fails to allocate MSI-X interrupts after 30-32 > vectors had been allocated. Setting higher number in > ddi_msix_alloc_limit does not resolve the issue. > > Should the driver use new dynamic Interrupt Resource Manager interface? > What is the reason for the above limitation? > > Please advice. > > Eitan > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > driver-discuss mailing list > driver-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/driver-discuss _______________________________________________ driver-discuss mailing list driver-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/driver-discuss
