Re: [vpp-dev] [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email
> 3n-hsw Around 2nd of April there happened a regression in eth-l2bdscale1mmaclrn test, visible here [0]. Trending needed multiple runs to identify it is there, and alerting is configured not to report "old" regressions (so it did not report this one). Anyway, the closer look shows previously the test had larger stdev, and even closer look shows there was small duration stretching going on. The regression than happened when [1] fixed it by adding TRex workers. Unintentionally, as it was fixing primarily Mellanox tests. See [2] for more details on that. Vratko. [0] https://docs.fd.io/csit/master/trending/trending/l2-3n-hsw-xl710.html#t1c [1] https://gerrit.fd.io/r/c/csit/+/26262 [2] https://lists.fd.io/g/csit-report/message/2406 -Original Message- From: csit-rep...@lists.fd.io On Behalf Of nore...@jenkins.fd.io Sent: Friday, 2020-April-03 18:07 To: Fdio+Csit-Report via Email Integration Subject: [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email Following regressions occured in the last trending job runs, listed per testbed type. 2n-skx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/881, VPP version: 20.05-rc0~433-g0c7aa7ab5~b930 No regressions 3n-skx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/862, VPP version: 20.05-rc0~433-g0c7aa7ab5~b930 tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-2t1c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-4t2c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-8t4c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.vm vhost.25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-8t4c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 2n-clx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/282, VPP version: 20.05-rc0~456-g57a5a2df5~b953 tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-4t2c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-ethip4-ip4base-eth-2memif-1dcr-mrr.64b-8t4c-avf-ethip4-ip4base-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp
Re: [vpp-dev] [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email
> 100ge2p1cx556a-rdma No VPP code change. Up to now, RDMA tests were suffering from "duration stretching". That is a phenomenon where TRex takes longer to send the required number of packets, so device under tests experiences smaller offered load and MRR value is inflated. We have suppressed the phenomenon by tweaking [0] core usage and max rate (pps) limit. The new numbers should be reliable now. Vratko. [0] https://gerrit.fd.io/r/c/csit/+/26262 -Original Message- From: csit-rep...@lists.fd.io On Behalf Of nore...@jenkins.fd.io Sent: Friday, 2020-April-03 18:07 To: Fdio+Csit-Report via Email Integration Subject: [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email Following regressions occured in the last trending job runs, listed per testbed type. 2n-skx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/881, VPP version: 20.05-rc0~433-g0c7aa7ab5~b930 No regressions 3n-skx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/862, VPP version: 20.05-rc0~433-g0c7aa7ab5~b930 tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-2t1c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-4t2c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-8t4c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr tests.vpp.perf.vm vhost.25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-8t4c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 2n-clx, CSIT build: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/282, VPP version: 20.05-rc0~456-g57a5a2df5~b953 tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-4t2c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-avf-ethip4-ip4base-eth-2memif-1dcr-mrr.64b-8t4c-avf-ethip4-ip4base-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-25ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2xcbase-eth-2memif-1dcr-mrr tests.vpp.perf.container memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-4t2c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr tests.v
