Re: [PATCH: 2.6.13-15-SMP 3/3] network: concurrentlyrunsoftirqnetwork code on SMP
On Wed, 2007-26-09 at 10:12 +0800, John Ye wrote: cpu hash (srcip + dstip) % nr_cpus, plus checking cpu balance periodically, shift cpu by an extra seed value? That may work maybe even add ipproto as a 3rd tuple; experiments will tell - you need to be able to generate a random enough traffic pattern. For example if the traffic pattern is always for your local host the dstip may not be useful to the hash. You could always try the hash tests outside first; look at some of the jenkins hashes in the kernel __do_IRQ has a tendency to collect same IRQ on different CPUs into one CPU when NIC is busy(by IRQ_PENDING IRQ_INPROGRESS control skill). so, dispatch the load to SMP here may be good thing(?). possibly. Or you may wanna tie the NIC IRQ to a CPU. cheers, jamal - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH: 2.6.13-15-SMP 3/3] network: concurrentlyrunsoftirqnetwork code on SMP
Jamal Stephen, I found BSS-hash paper you mentioned and have browsed it briefly. The issue may end sending all your packets to one cpu might be dealt with by cpu hash (srcip + dstip) % nr_cpus, plus checking cpu balance periodically, shift cpu by an extra seed value? Any way, the cpu hash code must not be too expensive because every incoming packet hits the path. We are going to do further study on this BSS thing. __do_IRQ has a tendency to collect same IRQ on different CPUs into one CPU when NIC is busy(by IRQ_PENDING IRQ_INPROGRESS control skill). so, dispatch the load to SMP here may be good thing(?). Thanks. John Ye - Original Message - From: jamal [EMAIL PROTECTED] To: Stephen Hemminger [EMAIL PROTECTED] Cc: john ye [EMAIL PROTECTED]; David Miller [EMAIL PROTECTED]; netdev@vger.kernel.org; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 6:22 AM Subject: Re: [PATCH: 2.6.13-15-SMP 3/3] network: concurrentlyrunsoftirqnetwork code on SMP On Tue, 2007-25-09 at 09:03 -0700, Stephen Hemminger wrote: There is a standard hash called RSS, that many drivers support because it is used by other operating systems. I think any stateless/simple thing will do (something along the lines what 802.1ad does for trunk, a 5 classical five tuple etc). Having solved the reordering problem in such a stateless way introduces a loadbalancing setback; you may end sending all your packets to one cpu (a problem Mr Ye didnt have when he was re-orderding ;-). cheers, jamal - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH: 2.6.13-15-SMP 3/3] network: concurrentlyrunsoftirqnetwork code on SMP
Dear Jamal, Thanks, bothered you all. I will look into the 2 issues. re-ordering and spinlock, and do extensive test. Once having result, no matter positive or negative, I will contact you. The format will not be a mess any more. John Ye - Original Message - From: jamal [EMAIL PROTECTED] To: john ye [EMAIL PROTECTED] Cc: David Miller [EMAIL PROTECTED]; netdev@vger.kernel.org; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, September 24, 2007 2:07 AM Subject: Re: [PATCH: 2.6.13-15-SMP 3/3] network: concurrentlyrunsoftirqnetwork code on SMP John, It will NEVER be an acceptable solution as long as you have re-ordering. I will look at it - but i have to run out for now. In the meantime, I have indented it for you to be in proper kernel format so others can also look it. Attached. cheers, jamal - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
