> -----Original Message----- > From: [email protected] [mailto:linux- > [email protected]] On Behalf Of Sasha > Sent: Wednesday, August 12, 2009 8:18 PM > To: Domsch, Matt > Cc: linux-poweredge-Lists > Subject: RE: megaraid_sas PERC 6 CPU consumption > > > > > ----------------------------------------------------------------------- > --- > This e-mail Contains PeerApp Proprietary and Confidential information. > > > ----------------------------------------------------------------------- > --- > -----Original Message----- > > From: Matt Domsch [mailto:[email protected]] > > Sent: ד 12 אוגוסט 2009 16:47 > > > > On Mon, Aug 10, 2009 at 08:37:03AM +0300, Sasha wrote: > > > We're using PERC 6 (MegaRAID SAS 1078) controller on R610 > machines. When > > > under stress, PERC interrupts (megasas in /proc/interrupts) > consume up to > > > 100% of CPU time on one of the cores. Usually this is CPU0, but > we can > > > move it by writing into /proc/<megasas IRQ number>/smp_affinity > file. I > > > was wondering if it is possible to reduce its CPU consumption, > or perhaps > > > spread the load among several cores. > > > > Using the smp_affinity, which is a bitmask, you can spread the > > interrupts across several cores, simply by setting the appropriate > CPU > > ID bits to 1. > > > > echo "0x55" > smp_affinity > > > > would send interrupts to CPUs 1, 3, 5, and 7. > > I am afraid this does not work for us on R610. > Writing a bitmask into smp_affinity file implies that IO-APIC has been > configured to work in so called Logical Low Priority interrupt delivery > mode. The problem is that on R610 kernel does not put IO-APIC into this > mode. This is because: > 1. We have two quad core CPUs and hyper-threading, meaning that we have > 16 cores. IO-APIC can address up to 8 (it has one byte bitmask for the > destination core, thus it can address up 8 cores). > 2. Local APIC ID of the CPUs is higher than 8 (even with hyper- > threading off). Since in logical mode IO-APIC can address up to 8 > cores, kernel drops it into so called Physical Fixed interrupt delivery > mode - i.e. every interrupt gets to single core. In this mode it can > address up to 256 cores (it’s the same one byte register, but this time > it is treated as a number, not a bitmask). > > I was thinking about something else. Network devices support MSI-X PCI > extension. It allows single device to have multiple interrupt vectors. > With multiple interrupt vectors per device, even in physical mode, we > can spread interrupts across several cores. For instance onboard > Broadcom NICs have 8 vectors per port, thus using the above > smp_affinity trick you can configure 8 cores to serve its interrupts. > I am wondering if there's something similar for megaraid_sas? >
Yes. The megaraid_sas driver does not support msix as yet including the upstream versions. _______________________________________________ Linux-PowerEdge mailing list [email protected] https://lists.us.dell.com/mailman/listinfo/linux-poweredge Please read the FAQ at http://lists.us.dell.com/faq
