Re: Why ping latency is smaller with shorter send interval?
Can you please not send the same posting to the mailing list three times, from three different email addresses? Once is enough, thank you.
Re: Why ping latency is smaller with shorter send interval?
Hi, Eric Dumazet writes: > > Many factors in play here. > > 1) if you keep cpus busy enough, they tend to keep in their caches > the data needed to serve your requests. In your case, time taken to > process an ICMP packet can be very different depending on how hot > cpu caches are. > > 2) Idle cpus can be put in a power conserving state. > It takes time to exit from these states, as you noticed. > These delays can typically be around 50 usec, or more. Still on the power management theme, in my experience, in addition to the CPU power states, the PCIe and NIC power management settings also effect latency on the order of 10-100s usecs when the system is allowed to go idle, some things that have helped: - setting CONFIG_PCIEASPM to performance; - disabling EEE (energy efficient ethernet) in your NIC; > > Search for cpu C-states , and powertop program. > Cheers, -- Vinicius
Re: Why ping latency is smaller with shorter send interval?
On Fri, Oct 02, 2020 at 11:26:00AM +0200, Eric Dumazet wrote: > 2) Idle cpus can be put in a power conserving state. > It takes time to exit from these states, as you noticed. > These delays can typically be around 50 usec, or more. This is often the case in my experience. I'm even used to starting a busy loop in SCHED_IDLE prio on certain machines or in certain VMs just to keep them busy, and to see the ping latency cut in half. Willy
Re: Why ping latency is smaller with shorter send interval?
On 10/2/20 10:51 AM, 叶小龙 wrote: > Hi, net experts, > > Hope this is the right place to ask the question :) > > Recently I've tried to measure the network latency between two > machines by using ping, one interesting observation I found is that > ping latency will be smaller if I use a shorter interval with -i > option. For example, > > when I use default ping (interval is 1s), then the ping result is as > below with avg latency 0.062ms > > # ping 9.9.9.2 -c 10 > PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data. > 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.059 ms > 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.079 ms > 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.060 ms > 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.072 ms > 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.048 ms > 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.069 ms > 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.067 ms > 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.055 ms > 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.058 ms > 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.055 ms > > --- 9.9.9.2 ping statistics --- > 10 packets transmitted, 10 received, 0% packet loss, time 9001ms > rtt min/avg/max/mdev = 0.048/0.062/0.079/0.010 ms > > Then I use "-i 0.001", the lateny (0.038) is way better than defaut ping > > # ping 9.9.9.2 -i 0.001 -c 10 > PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data. > 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.069 ms > 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.039 ms > 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.034 ms > 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.033 ms > 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.033 ms > 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.033 ms > 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.034 ms > 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.036 ms > 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.037 ms > 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.038 ms > > --- 9.9.9.2 ping statistics --- > 10 packets transmitted, 10 received, 0% packet loss, time 9ms > rtt min/avg/max/mdev = 0.033/0.038/0.069/0.012 ms > > > ping loopback shows the similar result. > > Default ping avg latency is 0.049ms > > # ping 127.0.0.1 -c 10 > PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. > 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.032 ms > 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.049 ms > 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.054 ms > 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.058 ms > 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.049 ms > 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.042 ms > 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.052 ms > 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.052 ms > 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.053 ms > 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.055 ms > > --- 127.0.0.1 ping statistics --- > 10 packets transmitted, 10 received, 0% packet loss, time 9001ms > rtt min/avg/max/mdev = 0.032/0.049/0.058/0.010 ms > > ping with "-i 0.001" shows 0.014ms avg latency. > > # ping 127.0.0.1 -i 0.001 -c 10 > PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. > 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.040 ms > 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.014 ms > 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.012 ms > 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.011 ms > 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.011 ms > 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.011 ms > 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.011 ms > 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.010 ms > 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.010 ms > 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.011 ms > > --- 127.0.0.1 ping statistics --- > 10 packets transmitted, 10 received, 0% packet loss, time 9ms > rtt min/avg/max/mdev = 0.010/0.014/0.040/0.008 ms > > I'm using centos 7.2 with kernel 3.10. > > I am very confused about the result. As I understand it, it doesn't > matter how frequently I send packets, each packet's latency should be > the same. So How can I understand it from network stack point of view? > > Any thoughts or suggestions would be highly appreciated. > > > Thanks, > Xiaolong > Many factors in play here. 1) if you keep cpus busy enough, they tend to keep in their caches the data needed to serve your requests. In your case, time taken to process an ICMP packet can be very different depending on how hot cpu caches are. 2) Idle cpus can be put in a power conserving state. It takes time to exit from these states, as you noticed. These delays can typically be around 50 usec, or more. Search for cpu C-states , and powertop program.
Why ping latency is smaller with shorter send interval?
Hi, net experts, Hope this is the right place to ask the question :) Recently I've tried to measure the network latency between two machines by using ping, one interesting observation I found is that ping latency will be smaller if I use a shorter interval with -i option. For example, when I use default ping (interval is 1s), then the ping result is as below with avg latency 0.062ms # ping 9.9.9.2 -c 10 PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data. 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.059 ms 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.079 ms 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.060 ms 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.072 ms 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.048 ms 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.069 ms 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.067 ms 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.055 ms 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.058 ms 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.055 ms --- 9.9.9.2 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9001ms rtt min/avg/max/mdev = 0.048/0.062/0.079/0.010 ms Then I use "-i 0.001", the lateny (0.038) is way better than defaut ping # ping 9.9.9.2 -i 0.001 -c 10 PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data. 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.069 ms 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.039 ms 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.034 ms 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.033 ms 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.033 ms 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.033 ms 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.034 ms 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.036 ms 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.037 ms 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.038 ms --- 9.9.9.2 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9ms rtt min/avg/max/mdev = 0.033/0.038/0.069/0.012 ms ping loopback shows the similar result. Default ping avg latency is 0.049ms # ping 127.0.0.1 -c 10 PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.032 ms 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.049 ms 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.054 ms 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.058 ms 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.049 ms 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.042 ms 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.052 ms 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.052 ms 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.053 ms 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.055 ms --- 127.0.0.1 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9001ms rtt min/avg/max/mdev = 0.032/0.049/0.058/0.010 ms ping with "-i 0.001" shows 0.014ms avg latency. # ping 127.0.0.1 -i 0.001 -c 10 PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.040 ms 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.014 ms 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.012 ms 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.011 ms 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.011 ms 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.011 ms 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.011 ms 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.010 ms 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.010 ms 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.011 ms --- 127.0.0.1 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9ms rtt min/avg/max/mdev = 0.010/0.014/0.040/0.008 ms I'm using centos 7.2 with kernel 3.10. I am very confused about the result. As I understand it, it doesn't matter how frequently I send packets, each packet's latency should be the same. So How can I understand it from network stack point of view? Any thoughts or suggestions would be highly appreciated. Thanks, Xiaolong