Rajagopal Kunhappan writes: > Shreyas wrote: > > Hello all, > > > > I face a similar problem of un-utilized CPUs. > > I have a Intel Xeon 2.66GHz, with 8 CPU cores. The machine has Opensolaris > > build 48. I am using a 10 Gig card, whose driver I have written, to run > > iperf at full blast. > > > > I see that utilization of one of the CPUs is 80-100% where as rest of them > > stay idle at 0-3 %. Earlier I thought, this was due to locking in my > > driver code. I modified the driver code to reduce time under locking to a > > great extent. I used 'lockstat -o output_file sleep 20' to verify this. > > But this did not improve the situation atall. The mpstat looks like > > following : > > > > CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt > > idl > > 0 0 0 0 1388 220 102 0 2 52 0 79 0 3 0 > > 97 > > 1 13 0 0 1079 0 76 0 1 7 0 111 0 1 0 > > 99 > > 2 0 0 0 1042 0 4 0 2 0 0 0 0 1 0 > > 99 > > 3 0 0 0 1054 4 0 0 0 0 0 0 0 0 0 > > 100 > > 4 0 0 6146 207 0 70 0 1 316 0 10 0 2 0 > > 98 > > 5 0 0 0 1068 3 30 0 2 0 0 0 0 0 0 > > 100 > > 6 0 0 1036 917 2 58 1 2 476 0 65 0 6 0 > > 94 > > 7 0 0 161 62340 61308 6363 2139 1 2442 0 530 0 99 0 > > 1 > > > > > > CPU(7) has 99% utilization. The throughput of iperf increases greatly when > > more CPUs are utilized. > > > > Can I somehow distribute the interrupt handling among all the CPUs > > available ? I have even used softirq (deffered work) in which I complete > > most of the interrupt handling work. Can I attach the softirq routine to a > > CPU when I trigger it ? Or can I tweak some system parameters ? The intrd > > distribution of interrupts does not seem to work with me :( > > > > > > /var/adm/messages show this message : > > > > Dec 27 19:16:12 opensol-48 unix: [ID 954099 kern.info] NOTICE: IRQ16 is > > being shared by drivers with different interrupt levels. > > Dec 27 19:16:12 opensol-48 This may result in reduced system performance. > > > > I have tried to change the interrupt level of my network driver interrupt > > handler, but ddi_intr_set_pri() returns failure for any level other than > > 6. I am using legacy interrupts as I could not initialize MSI interrupts. > > Will MSI/X interrupts help avoid this sharing of interrupt line ? > > > > Do I need to do anything in driver to take advantage of soft rings from dl > > ? (it seems I do not need to do anything) > > > If your driver is a nemo (gldv3) driver, then setting ip_squeue_fanout > to 1 will use soft rings. > By default, 2 soft rings are created. You can create more by setting > ip_soft_rings_cnt.
Raja, does he also need to advertise the capability using DL_CAPAB_SOFT_RING ? -r _______________________________________________ networking-discuss mailing list [email protected]
