Hi Abeeha, In the config file, hqos thread is allocated (cpu section), and hqos is’nt enabled on interfaces (dpdk section). In the current state, hqos needs separate thread (other than worker threads) for scheduling function, and also, enablement on dpdk interfaces.
Thanks, Jasvinder From: Abeeha Aqeel [mailto:abeeha.aq...@xflowresearch.com] Sent: Wednesday, May 15, 2019 11:03 AM To: Singh, Jasvinder <jasvinder.si...@intel.com>; Ni, Hongjun <hongjun...@intel.com> Cc: Byrne, Stephen1 <stephen1.by...@intel.com>; Dumitrescu, Cristian <cristian.dumitre...@intel.com>; vpp-dev@lists.fd.io; b...@xflowresearch.com Subject: RE: [vpp-dev] HQoS Hi Jasvinder, Here’s the startup.conf: unix { nodaemon log /var/log/vpp/vpp.log full-coredump cli-listen /run/vpp/cli.sock # gid vpp } api-trace { on } # api-segment { # gid vpp # } # socksvr { # default # } cpu { main-core 2 corelist-workers 4, 20, 6, 22, 8, 24 # corelist-hqos-threads 18 } dpdk { dev 0000:05:00.0 { num-rx-queues 2 # hqos } dev 0000:05:00.1 { num-rx-queues 2 # hqos } # igb_uio # uio_pci_generic ## Disable multi-segment buffers, improves performance but ## disables Jumbo MTU support # no-multi-seg ## Change hugepages allocation per-socket, needed only if there is need for ## larger number of mbufs. Default is 256M on each detected CPU socket socket-mem 16384, 16384 ## Disables UDP / TCP TX checksum offload. Typically needed for use ## faster vector PMDs (together with no-multi-seg) # no-tx-checksum-offload } plugins { path /root/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/ plugin default { enable } } I tried enabling and disabling hqos but it doesn’t work in both cases. Regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Singh, Jasvinder<mailto:jasvinder.si...@intel.com> Sent: Wednesday, May 15, 2019 2:29 PM To: Abeeha Aqeel<mailto:abeeha.aq...@xflowresearch.com>; Ni, Hongjun<mailto:hongjun...@intel.com> Cc: Byrne, Stephen1<mailto:stephen1.by...@intel.com>; Dumitrescu, Cristian<mailto:cristian.dumitre...@intel.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; b...@xflowresearch.com<mailto:b...@xflowresearch.com> Subject: RE: [vpp-dev] HQoS Hi Abeeha, Can you share startup.conf? Thanks, Jasvinder From: Abeeha Aqeel [mailto:abeeha.aq...@xflowresearch.com] Sent: Wednesday, May 15, 2019 7:29 AM To: Singh, Jasvinder <jasvinder.si...@intel.com<mailto:jasvinder.si...@intel.com>>; Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> Cc: Byrne, Stephen1 <stephen1.by...@intel.com<mailto:stephen1.by...@intel.com>>; Dumitrescu, Cristian <cristian.dumitre...@intel.com<mailto:cristian.dumitre...@intel.com>>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; b...@xflowresearch.com<mailto:b...@xflowresearch.com> Subject: RE: [vpp-dev] HQoS Hi Jasvinder, The CMakelist.txt files before and after applying the patch are exactly the same and hqos plugin is already enabled but the CLI commands are still not showing. Below is the screenshot of the CMakelist.text file located at /root/vpp/src/plugins/dpdk: [cid:image002.png@01D50B08.CB3C2CA0] Thank you and best regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Abeeha Aqeel<mailto:abeeha.aq...@xflowresearch.com> Sent: Wednesday, May 15, 2019 11:04 AM To: hongjun...@intel.com<mailto:hongjun...@intel.com> Cc: stephen1.by...@intel.com<mailto:stephen1.by...@intel.com>; cristian.dumitre...@intel.com<mailto:cristian.dumitre...@intel.com>; cristian.dumitre...@intel.com<mailto:cristian.dumitre...@intel.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; b...@xflowresearch.com<mailto:b...@xflowresearch.com> Subject: FW: [vpp-dev] HQoS Hi Jasvinder, The CMakelist.txt files before and after applying the patch are exactly the same and hqos plugin is already enabled but the CLI commands are still not showing. Below is the screenshot of the CMakelist.text file located at /root/vpp/src/plugins/dpdk: [cid:image002.png@01D50B0D.103CAD30] Thank you and best regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Singh, Jasvinder<mailto:jasvinder.si...@intel.com> Sent: Friday, May 10, 2019 2:58 PM To: Ni, Hongjun<mailto:hongjun...@intel.com>; Abeeha Aqeel<mailto:abeeha.aq...@xflowresearch.com> Cc: Byrne, Stephen1<mailto:stephen1.by...@intel.com>; Dumitrescu, Cristian<mailto:cristian.dumitre...@intel.com> Subject: RE: [vpp-dev] HQoS + Cristian From: Singh, Jasvinder Sent: Friday, May 10, 2019 10:52 AM To: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>>; Abeeha Aqeel <abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com>> Cc: Byrne, Stephen1 <stephen1.by...@intel.com<mailto:stephen1.by...@intel.com>> Subject: RE: [vpp-dev] HQoS Hi Abeeha, Looks like HQoS module is disabled in current VPP dpdk plugin (Check CMakelist.txt). Please trying enabling it first. You should see HQoS related CLIs on the console. Thanks, Jasvinder From: Ni, Hongjun Sent: Friday, May 10, 2019 2:47 AM To: Abeeha Aqeel <abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com>>; Singh, Jasvinder <jasvinder.si...@intel.com<mailto:jasvinder.si...@intel.com>> Subject: RE: [vpp-dev] HQoS Hi Jasvinder, Could you help to look into this issue? Thank you! Thanks, Hongjun From: Abeeha Aqeel [mailto:abeeha.aq...@xflowresearch.com] Sent: Thursday, May 9, 2019 6:02 PM To: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>>; Singh, Jasvinder <jasvinder.si...@intel.com<mailto:jasvinder.si...@intel.com>> Cc: b...@xflowresearch.com<mailto:b...@xflowresearch.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: RE: [vpp-dev] HQoS Hi Hongjun, I applied the patch submitted to fix the API change on vpp 19.04 using underlying dpdk 19.02, following the steps: [cid:image001.png@01D5071E.C2A73910] git clone https://gerrit.fd.io/r/vppHi Hongjun, cd vpp make wipe make install-dep make install-ext-deps wget https://gerrit.fd.io/r/changes/16839/revisions/a62db7f3796ef152d23475ed36ad5f3fbfcab2a8/archive?format=tar mv archive\?format\=tar a62db7f.tar tar -xvf a62db7f.tar make build ./vpp -c /root/go/src/github.com/xFlowResearch/BNG/vpp-startup.conf & ./vppctl I also tried, git fetch and cherry-pick: cd vpp make wipe make install-dep make install-ext-deps git stash git fetch https://gerrit.fd.io/r/vpp refs/changes/39/16839/3 && git cherry-pick FETCH_HEAD make build ./vpp -c /root/go/src/github.com/xFlowResearch/BNG/vpp-startup.conf & ./vppctl In both cases, the setup builds and starts properly but hqos plugin doesn’t work. Nor does the vpp debug CLI have the hqos commands, e.g. in this case it should have show dpdk interface hqos placement etc. [cid:image002.png@01D5071E.C2A73910] I am working on Centos 7 and have assigned 1024 2MB hugepages. I have also attached my vpp configuration file with this email. Please suggest what could be wrong and why the plugin is not loading properly. Regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Abeeha Aqeel<mailto:abeeha.aq...@xflowresearch.com> Sent: Thursday, May 9, 2019 10:10 AM To: Ni, Hongjun<mailto:hongjun...@intel.com>; Singh, Jasvinder<mailto:jasvinder.si...@intel.com> Cc: b...@xflowresearch.com<mailto:b...@xflowresearch.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] HQoS Hi Hongjun, Yes, sure. I will try the patch and share my results as soon as possible. Regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Ni, Hongjun<mailto:hongjun...@intel.com> Sent: Tuesday, May 7, 2019 6:40 PM To: Abeeha Aqeel<mailto:abeeha.aq...@xflowresearch.com>; Singh, Jasvinder<mailto:jasvinder.si...@intel.com> Cc: b...@xflowresearch.com<mailto:b...@xflowresearch.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] HQoS Hi Abeeha, There is a patch submitted to fix the API change. https://gerrit.fd.io/r/#/c/16839/ Could you help to give it a try? Thanks, Hongjun From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> [mailto:vpp-dev@lists.fd.io] On Behalf Of Abeeha Aqeel Sent: Monday, May 6, 2019 6:32 PM To: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> Cc: b...@xflowresearch.com<mailto:b...@xflowresearch.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] HQoS Hi Hongjun, I have tested the HQoS Plugin with iperf3. Below is a simple topology I have implemented: [cid:image001.png@01D5010B.5C565F10] First, I tested the default profile (profile 0) provided by VPP and assigned it to all 4096 pipes (users) which should give approximately 2MB (10G/4096) to each user. It worked fine for 5 users i.e. each user was assigned 2MB but for an increased number of users it didn’t work the same way. I changed the token bucket rate, token bucket size and tc_period parameters in the hqos.c code which didn’t give me any concrete results. I have attached results for 10 clients with different combinations of token bucket size, token bucket rate and tc_period. Any ideas what the problem might be ? Also, in what proportion should the tb_rate, tb_size , tc_rate and tc_period be assigned? S# Rate (Bytes per second) Token Bucket Size (Bytes) TC Period (ms) Total no of PPPoE sessions Expected Result (Cumulative average bandwidth in Mbps) Actual Result (Cumulative average bandwidth in Mbps) Comments Subport Pipe Subport Pipe Subport Pipe 1 1250000000 305175 1000000 1000000 10 40 10 24.4 8.1 vpp remains stable 2 1250000000 305175 1000000 1000000 10 60 10 24.4 12.72 vpp remains stable 3 1250000000 305175 1000000 1000000 10 100 10 24.4 11.4 vpp remains stable 4 1250000000 305175 1000000 10000000 10 40 10 24.4 12.1 vpp remains stable 5 1250000000 305175 10000000 10000000 10 40 10 24.4 12.03 vpp remains stable 6 1250000000 305175 10000000 10000000 10 100 10 24.4 2.21 vpp crashes after a few seconds 7 1250000000 305175 1000000 1000000 10 60 10 24.4 12.1 vpp remains stable Regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05] From: Ni, Hongjun<mailto:hongjun...@intel.com> Sent: Monday, May 6, 2019 12:52 PM Subject: RE: [vpp-dev] HQoS Hi Abeeha, For downstream bandwidth limiting, we leveraged HQos plugin in OpenBRAS solution. In our previous integration test, it could support 64K subscribers with HQos. Thanks, Hongjun From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> [mailto:vpp-dev@lists.fd.io] On Behalf Of Abeeha Aqeel Sent: Monday, May 6, 2019 3:28 PM To: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> Cc: b...@xflowresearch.com<mailto:b...@xflowresearch.com>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: [vpp-dev] HQoS Hi Hongjun, I have been trying to implement downstream bandwidth limiting using HQoS plugin in vpp. It works fine for a certain number of clients (less than 5) but doesn’t assign proper bandwidth for larger number of clients. Can you please elaborate which method is being used in the OpenBRAS solution for band limiting traffic? Are you using the token-bucket algorithm as well? Regards, Abeeha Aqeel Network Design Engineer Xflow Research Inc. +923245062309 (GMT+5) abeeha.aq...@xflowresearch.com<mailto:abeeha.aq...@xflowresearch.com> www.xflowresearch.com<http://www.xflowresearch.com/> [cid:6e5d1814-0a3d-444b-9050-060db1465f05]
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13043): https://lists.fd.io/g/vpp-dev/message/13043 Mute This Topic: https://lists.fd.io/mt/31517544/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
