Keegan, policer configuration is shown here: http://img690.imageshack.us/img690/3162/iperftest.png
Sthaug, thanks for explanation! So basically one might describe small buffer vs larger buffer with following drawing: http://img24.imageshack.us/img24/3777/leakybucket.png (I'm not very good with Gimp :P ) Water flows our from both buckets(the 1024KB one and the 512KB one) with consistent rate of 10Mbps. If water is poured into the buckets with inconsistent rate, the larger buffer is less sensitive to bursts while in case of a small buffer, high burst might cause water to overflow(packet loss). Chris, with such policer: firewall { policer bw-10Mbps { filter-specific; if-exceeding { bandwidth-limit 10m; burst-size-limit 512k; } then discard; } } ..results between 192.168.1.1 and 192.168.2.1 were following: IPERF: # iperf -c 192.168.2.1 -u -fm -t60 -d -b 10m ------------------------------------------------------------ Server listening on UDP port 5001 Receiving 1470 byte datagrams UDP buffer size: 0.04 MByte (default) ------------------------------------------------------------ ------------------------------------------------------------ Client connecting to 192.168.2.1, UDP port 5001 Sending 1470 byte datagrams UDP buffer size: 0.01 MByte (default) ------------------------------------------------------------ [ 4] local 192.168.1.1 port 47344 connected with 192.168.2.1 port 5001 [ 3] local 192.168.1.1 port 5001 connected with 192.168.2.1 port 38624 [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 71.5 MBytes 10.0 Mbits/sec [ 4] Sent 51021 datagrams [ 3] 0.0-60.0 sec 69.9 MBytes 9.77 Mbits/sec 0.034 ms 1186/51021 (2.3%) [ 3] 0.0-60.0 sec 1 datagrams received out-of-order [ 4] Server Report: [ 4] 0.0-60.0 sec 69.9 MBytes 9.77 Mbits/sec 0.126 ms 1192/51020 (2.3%) [ 4] 0.0-60.0 sec 1 datagrams received out-of-order # NUTTCP: [root@ ~]# nuttcp -4 -u -Ri10M -T1m -v 192.168.2.1 nuttcp-t: v6.1.2: socket nuttcp-t: buflen=1024, nstream=1, port=5001 udp -> 192.168.2.1 nuttcp-t: time limit = 60.00 seconds nuttcp-t: rate limit = 10.000 Mbps (instantaneous), 1220 pps nuttcp-t: send window size = 9216, receive window size = 42080 nuttcp-t: 71.2344 MB in 60.12 real seconds = 1213.27 KB/sec = 9.9391 Mbps nuttcp-t: 72951 I/O calls, msec/call = 0.84, calls/sec = 1213.39 nuttcp-t: 9.1user 50.2sys 1:00real 98% 106i+1046d 490maxrss 0+1pf 0+11723csw nuttcp-r: v6.1.2: socket nuttcp-r: buflen=1024, nstream=1, port=5001 udp nuttcp-r: send window size = 109568, receive window size = 109568 nuttcp-r: available send window = 82176, available receive window = 82176 nuttcp-r: 69.4453 MB in 60.12 real seconds = 1182.79 KB/sec = 9.6894 Mbps nuttcp-r: 1832 / 72944 drop/pkt 2.51% data loss nuttcp-r: 71116 I/O calls, msec/call = 0.87, calls/sec = 1182.86 nuttcp-r: 0.0user 0.4sys 1:00real 0% 0i+0d 0maxrss 0+6pf 71045+2csw [root@ ~]# So basically the packet loss was pretty much the same with both utilities. However, in order to wrap this "Juniper "firewall policer" inner workings" topic up, the main thing is that while Juniper counts the network layer header into the 10Mbps policer, the iperf/nuttcp don't :) regards, martin 2011/4/9 Chris Tracy <ctr...@es.net>: > Hi Martin, > >> Iperf UDP is indeed very bursty- I ran "iperf -c 192.168.2.1 -u -fm >> -t60 -d -b 10m" and at the same time did "tcpdump -w iperf_dump.pcap >> host 192.168.2.1". Then printed deltas <...> > > Yes, as you can see from these results, the majority of the packets are > spaced very close (~11-12 microseconds apart). > > If you just look at the first few hundred lines of the tcpdump -ttt output, > you'll also see the burst / wait / burst / wait / burst pattern. > >> However, if I ran "nuttcp -4 -u -Ri10M -v -T1m -w2m 192.168.2.1" and >> at the same time did "tcpdump -w nuttcp_dump.pcap host 192.168.2.1", >> the most popular delta values between packets are way higher than with Iperf: >> ... >> As you said, in case of nuttcp UDP flood, packets are much more evenly sent >> :) > > Right, and you can make nuttcp more bursty by doing -Ri10M/5 or -Ri10M/10 to > instruct it to burst 5 or 10 packets at a time, respectively. By default, > nuttcp will pace packets as evenly as possible at the expense of burning a > core. Keep in mind that the behavior of iperf more closely emulates a > typical application -- I don't think users would find it very acceptable to > burn up their CPU just to do precise rate-limiting. :-) > > So, did you lose less packets through your policer when testing with nuttcp > vs iperf, assuming the same policer settings in the middle? > >> Could you explain this "Your interface speed appears to be GigE, so >> you are bursting out at line rate and relying on the buffers in the >> router to accommodate these line-rate bursts. You should have >> different results if you connected the host at FastE vs GigE." a bit >> further? I mean why should I see different results with 100BASE-TX or >> 10BASE-T? > > A picture is worth a thousand words....unfortunately, I have no picture to > explain this, but one would sure be useful. > > NICs either send out packets at line-rate or not -- it's either on or off. > By instructing nuttcp/iperf to "rate-limit" to 10Mbps, you're really saying, > "please don't send more than 1,250,000 bytes every second". As you can see > from the differences between iperf/nuttcp, _how_ an application handles this > request depends on its implementation. On the wire, frames are always sent > at line-rate -- if the application sends a burst in a short period of time, > the next device needs to have enough buffer to handle the burst as it > processes and forwards. Imagine you have a 10GigE-to-GigE speed conversion > somewhere along a path, sending host is connected at 10GigE and the receiving > host is connected at GigE. Many smaller devices do not have enough buffer > to handle an incoming bursty 10G flow destined to a 1G port, and you will see > a lot of drops -- but nuttcp might not observe any UDP loss at all on this > path because of the way it does its rate-limiting. No policing or shaping in > my example, but you're basically doing the same thing by putting a 10Mbps > bottleneck in a path where frames are originating from a host connected at > GigE. I think you can see from my example -- at a high-level -- that a > device receiving a burst of X bytes from a host connected at GigE has less > time (requiring more buffer) to process and forward that data than if the > host is connected at 100bT or 10bT (it will take 10 to 100X longer to send > the same number of bytes...) > >> In addition, how to start nuttcp in both directions simultaneously? >> "nuttcp -4 -u -Ri10M -v -T1m -w2m 192.168.2.1" only sends from my >> nuttcp client to nuttcp server. > > There is no equivalent of iperf -d (--dualtest) in nuttcp. You have two > options: > > 1) start 2 servers, one on each side, then kick off a client from each side > at around the same time > > 2) use the '-r' option to reverse the direction of the test, and kick off two > tests simultaneously (only requires one server running): nuttcp -4 -u -Ri10M > -v -T1m -w2m 192.168.2.1 & nuttcp -4 -u -Ri10M -v -T1m -w2m -r 192.168.2.1 > > You may also find it helpful to add '-i1' which will give you # of packets > lost every second, so you can see whether you are losing 1pps or a bursts of > 50 packets and clean in between. > > -Chris > > -- > Chris Tracy <ctr...@es.net> > Energy Sciences Network (ESnet) > Lawrence Berkeley National Laboratory > > _______________________________________________ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
