Cool, also would be glad to hear from the community :p
zhang...@yunshan.net.cn 发件人: 张东亚 发送时间: 2017-06-22 13:30 收件人: zhang...@yunshan.net.cn 抄送: 雷彦章; vpp-dev 主题: Re: Re: [vpp-dev]答复: 【vpp-dev】delay is error in ping with multi worker thread Hi zhangpan, Glad to here feedback, as I written in my previous mail, after we modify the code to using RPC mechanism, ping works fine but we have random 10ms issue while our normal ping delay is <1ms, we think the reason it due to schedule delay in the main thread but we want the community to confirm that. For vector processing, I think there is a upper limit for wait packets in order to reduce packet processing delay, it should not wait that much for a batch. 2017-06-22 13:11 GMT+08:00 zhang...@yunshan.net.cn <zhang...@yunshan.net.cn>: I didn't investigate the code but there's an assumption I would like to share. I encountered the same problem before, but what interested me is that I found if you send multiple ping pkts as fast as you can, the problem went away. I tried different numbers and if it's a multiple of 4, VPP works just as expected. I just assumed that in order to perform “vector processing”, VPP first waits for enough( > 4) pkts to store in a vector and then, process the batch. If you send ICMP pkt one by one, VPP will only start to process these pkts after receiving at lease 4 pkts. So for the first ping, the latency is 1000ms. Best Regards, Pan zhang...@yunshan.net.cn 发件人: 张东亚 发送时间: 2017-06-22 11:09 收件人: 雷彦章 抄送: vpp-dev 主题: Re: [vpp-dev]答复: 【vpp-dev】delay is error in ping with multi worker thread Hi List, I am colleague of Leiyanzhang, let me describe share the issues and investigations found when we test the ping command under multiple worker thread case. Stage1: Issue: Ping does not work under multiple worker thread case. Version: Vanilla vpp 17.04 (vanilla vp) Stage2: Issue: Ping got > 1s delay result under multiple worker thread case, 1s is the default ping interval, if we specify 10s interval, the delay will be 10s delay. Version: Vanilla vpp 17.04 + VPP-844 patch. Our Solution: as Lei pointed out, we think it's not correct for the worker thread to directly using the ping_run_t's cli_process_id to notify the main thread because the vlib_process_signal_event is using work thread vlib_main structure, however since process is shared by main thread and worker, when cli process in main thread timeout waiting event, it can still fetch the event, but the delay will be the interval. State3: Issue: Ping delay is not related with interval, but got 10ms+ delay randomly. Version: Vanilla vpp 17.04 + VPP-844 Patch + change to using vl_api_rpc_call_main_thread to notify cli process in main thread. Our solution: After read vpp code, we think worker thread should use vl_api_rpc_call_main_thread to notify cli process in main thread, after will made the fix, the ping command works as expected most time with only random 10ms+ delay. For this problem, we found the reason is related to vl_api_rpc_call_main_thread only put then event to master thread vl_input_queue, but does not notify and wake up master thread timely, the main thread can only process this when linux_epoll_input returns, while it will randomly sleep 10e3, if we change this value, the random ping delay will change too. We want to hear feedback from the list and hope we can find a better way to fix this.
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev