Hello all, I face a similar problem of un-utilized CPUs. I have a Intel Xeon 2.66GHz, with 8 CPU cores. The machine has Opensolaris build 48. I am using a 10 Gig card, whose driver I have written, to run iperf at full blast.
I see that utilization of one of the CPUs is 80-100% where as rest of them stay idle at 0-3 %. Earlier I thought, this was due to locking in my driver code. I modified the driver code to reduce time under locking to a great extent. I used 'lockstat -o output_file sleep 20' to verify this. But this did not improve the situation atall. The mpstat looks like following : CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 0 1388 220 102 0 2 52 0 79 0 3 0 97 1 13 0 0 1079 0 76 0 1 7 0 111 0 1 0 99 2 0 0 0 1042 0 4 0 2 0 0 0 0 1 0 99 3 0 0 0 1054 4 0 0 0 0 0 0 0 0 0 100 4 0 0 6146 207 0 70 0 1 316 0 10 0 2 0 98 5 0 0 0 1068 3 30 0 2 0 0 0 0 0 0 100 6 0 0 1036 917 2 58 1 2 476 0 65 0 6 0 94 7 0 0 161 62340 61308 6363 2139 1 2442 0 530 0 99 0 1 CPU(7) has 99% utilization. The throughput of iperf increases greatly when more CPUs are utilized. Can I somehow distribute the interrupt handling among all the CPUs available ? I have even used softirq (deffered work) in which I complete most of the interrupt handling work. Can I attach the softirq routine to a CPU when I trigger it ? Or can I tweak some system parameters ? The intrd distribution of interrupts does not seem to work with me :( /var/adm/messages show this message : Dec 27 19:16:12 opensol-48 unix: [ID 954099 kern.info] NOTICE: IRQ16 is being shared by drivers with different interrupt levels. Dec 27 19:16:12 opensol-48 This may result in reduced system performance. I have tried to change the interrupt level of my network driver interrupt handler, but ddi_intr_set_pri() returns failure for any level other than 6. I am using legacy interrupts as I could not initialize MSI interrupts. Will MSI/X interrupts help avoid this sharing of interrupt line ? Do I need to do anything in driver to take advantage of soft rings from dl ? (it seems I do not need to do anything) Thanks Shreyas This message posted from opensolaris.org _______________________________________________ perf-discuss mailing list [email protected]
