Hi, Checking every single FAIL job is very labour intensive - for humans. And we don’t have AI here, yet. Hence we’re wondering how to scale this operation. Proposal is to distribute the load. Here first attempt - build history list of 30 last jobs on LF jenkins page (older ones are purged) - correlated to gerrit triggers. High-level diagnostics if patten of failed tests easy to correlate to code patch. In the meantime CSIT team will keep improving high-level message reported per testcase in /<job-id>/robot/ (e.g. https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-master/1987/robot/ )
Shoot here how many I got wrong below - and you believe CSIT code is generating false positive FAILs - we want to hunt them down !! ##Anomalies - same gerrit patch - generating FAIL and PASS: ### None visible at the moment ##Other FAIL job runs - sorted by patch: ### patch issue? ### msg: TC FAIL - L2BD ### changes: vnet/vnet/l2/l2_input.h, vnet/vnet/l2/l2_input.c Failed > Console Output#1986 Oct 28, 2016 2:08 PM 3614,2 Failed > Console Output#1971 Oct 27, 2016 8:10 PM 3614,2 Failed > Console Output#1970 Oct 27, 2016 7:43 PM 3614,2 Failed > Console Output#1969 Oct 27, 2016 7:16 PM 3614,1 ### patch issue? ### msg: Parent suite setup failed: Could not pack testing framework. ### changes: many in vnet/*, vpp-api-test/*, vpp/vpp-api/* Failed > Console Output#1978 Oct 28, 2016 10:13 AM 3364,12 ### patch issue? ### msg: Socket timeout: ip_add_del_route ### changes: no idea, patch DRAFT - a PRIVATE SECRET CLEARLY :) Failed > Console Output#1967 Oct 27, 2016 2:50 PM 3533,8 ### patch issue? ### msg: TC FAIL - many, build failures ### changes: many in plugins/*, vnet/*, vpp/vpp-api/api.c<https://gerrit.fd.io/r/#/c/3445/13/vpp/vpp-api/api.c> Failed > Console Output#1983 Oct 28, 2016 12:45 PM 3445,18 Failed > Console Output#1980 Oct 28, 2016 11:36 AM 3445,17 Failed > Console Output#1976 Oct 28, 2016 10:01 AM 3445,16 Failed > Console Output#1975 Oct 28, 2016 8:20 AM 3445,15 Failed > Console Output#1966 Oct 27, 2016 1:38 PM 3445,14 Failed > Console Output#1965 Oct 27, 2016 12:49 PM 3445,13 Failed > Console Output#1961 Oct 27, 2016 11:21 AM 3445,12 ### patch issue - confirmed ### TC FAIL - DHCP ### changes: vnet/vnet/dhcpv6/proxy_error.def<https://gerrit.fd.io/r/#/c/3501/10/vnet/vnet/dhcpv6/proxy_error.def>, vnet/vnet/dhcpv6/proxy_node.c Success > Console Output#1987 Oct 28, 2016 2:47 PM 3501,10 Failed > Console Output#1981 Oct 28, 2016 12:15 PM 3501,9 Failed > Console Output#1960 Oct 27, 2016 9:07 AM 3501,9 ### patch issue - confirmed ### All TC FAIL - due to API failures? ### changes: vnet/vnet/dhcpv6/proxy_error.def<https://gerrit.fd.io/r/#/c/3501/10/vnet/vnet/dhcpv6/proxy_error.def>, vnet/vnet/dhcpv6/proxy_node.c Success > Console Output#1979 Oct 28, 2016 11:03 AM 3199,9 Failed > Console Output#1977 Oct 28, 2016 10:05 AM 3199,8 Success > Console Output#1974 Oct 28, 2016 6:45 AM 3199,7 Success > Console Output#1973 Oct 28, 2016 6:40 AM 3199,6 ### patch issue - confirmed ### TC FAIL - VM ### changes: vnet/vnet/devices/virtio/vhost-user.h, vnet/vnet/devices/virtio/vhost-user.c Success > Console Output#1985 Oct 28, 2016 2:04 PM 2922,21 Failed > Console Output#1964 Oct 27, 2016 12:17 PM 2922,20 Failed > Console Output#1963 Oct 27, 2016 12:13 PM 2922,19 ### Other PASS job runs: Success > Console Output#1984 Oct 28, 2016 1:35 PM 3621,1 Success > Console Output#1982 Oct 28, 2016 12:34 PM 3617,1 Success > Console Output#1972 Oct 28, 2016 2:14 AM 3602,2 Success > Console Output#1968 Oct 27, 2016 3:25 PM 3612,1 Success > Console Output#1962 Oct 27, 2016 11:40 AM 3318,5 Success > Console Output#1959 Oct 27, 2016 8:51 AM 3606,1 Success > Console Output#1958 Oct 27, 2016 8:48 AM 3605,1 -Maciek
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
