On 2018-05-16 18:06, Brian Rak wrote: > We've been trying to track down why our 5100's are dropping traffic > due to lack of buffer space, even with very low link utilization.
There's only 12 Mbyte of buffer space on the Trident II chip. If you get 10 Gbit/s bursts simultaneously on two ports, contending for the same output port, it will only take 10 ms to fill 12 Mbyte. (And of those 12 Mbyte, 3 Mbyte is used for dedicated per-port buffers, so you really only have ~9 Mbyte, so you would actually fill your buffers in 7.5-8 ms.) Do you see any actual problems due to the dropped packets? Some people would have you believe that TCP suffers horribly from a single dropped packet, but reality is not quite that bad. So don't chase problems that aren't there. Our busiest ports have drop rates at about 1 in every 15'000 packets (average over a few months), and so far we haven't noticed any TCP performance problems related to that. (But I should note that most of our traffic is long-distance, to and from sites at least several milliseconds away from us, and often a 10-20 ms away.) That said, for Trident II / Tomahawk level of buffer sizes, I think it makes sense to configure them to have it all actually used, and not wasted on the lossless queues. You should probably also consider enabling cut-through forwarding, if you haven't already done so. That should decrease the amount of buffer space used, leaving more available for when contention happens. /Bellman
signature.asc
Description: OpenPGP digital signature
_______________________________________________ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp