Yes, if you see kernel messages about neighbor table overflow, that could be a cause of general network unhappiness and excessive arps.
From: Mike Lovell <[email protected]> To: [email protected], Date: 03/27/2012 03:43 PM Subject: Re: [xcat-user] ARP traffic On 03/27/2012 08:19 AM, Baker D.J. wrote: Hello, We’re running xcat Version 2.6.10 on our cluster, and I’ve been taking a look at the network with tcpdump recently. We have in the region of 1000 compute nodes on the network, and they’re all managed by xcat. The volume of ARP related traffic on the network (I ran tcpdump on the xcat master) seems very high to me, however I’m no expert in these things and so the level of ARP activity may be reasonable. I ran tcpdump for just 1 minute, and recorded in the region of 7,000 ARP transactions. Is this reasonable for a cluster of 1000 compute nodes? If anyone can please advise us in this respect then that would be appreciated. Best regards – David. i wouldn't be too surprised if 7000 arps/sec is normal in this case. one thing you might want to look at changing is the values around how big the arp table can be. 1000 addresses is more than the default settings for the kernel to keep in the arp table for more than 30 seconds. so it might be that everything is constantly re-arp'ing cause the arp table is getting cleaned out. http://linux.die.net/man/7/arp has some info in the /proc interfaces section. you should look at the gc_interval and gc_thresh {1,2,3}. it might be a long shot but i once managed a system that had ~2000 devices on one broadcast domain and he had to up the sizes of gc_thresh {1,2,3} to get it so things played nicely. mike ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
<<inline: graycol.gif>>
------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
