On Wed, 2006-21-06 at 09:52 +1000, Herbert Xu wrote: > Well my gut feeling is that multiple qdisc_run's on the same dev can't > be good for perfomance. The reason is that SMP is only good when the > CPUs work on different tasks. If you get two or more CPUs to work on > qdisc_run at the same time they can still only supply one skb to the > device at any time. What's worse is that they will now have to fight > over the two spin locks involved which means that their cache lines > will bounce back and forth.
1) If the CPUs collide all the time it is without a doubt it is a bad thing (you can tell from tx_collission stats). 2) If on the other hand, the iota that a CPU enters that path in the softirq it gets the txlock then there is benefit to not serialize at the level you have done with that patch - you are enlarging the granularity of the serialization so much so that the CPU wont even get the opportunity to try and grab tx lock because it finds qdisc is already running. Your gut feeling is for #1 and my worry is for #2 ;-> I actually think your change is obviously valuable for scenarios where the bus is slower and therefore transmits take longer - my feeling is it may not be beneficial for fast buses like PCI-E or high speed PCI/X where the possibility of getting access tx collision is lower. The other reason I mentioned earlier as justification to leave the granularity at the level where it was is for good qos clocking. i.e to allow incoming packets to be used to clock the tx path - otherwise you will be dependent on HZ for your egress rate accuracy. I am not sure if this later point made sense - I could elaborate. The experiment needed to prove things is not hard: one needs to get a 2 or 4way machine and create a "funneling" effect to one NIC. For forwarding, the best setup will be to have 3 NICs. packets coming in on 2 NICs are forwarded to a third. The incoming-packet NICS are tied to different CPUs. In a 4way, the outgoing as well is tied to its own CPU. You then pummel the two incoming CPUs with pktgen or otherwise at something like 1Mpps (which is higher than the wire rate the third nic can handle). cheers, jamal - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
