Patch applied upstream Sending patches for Ubuntu kernel: https://lists.ubuntu.com/archives/kernel-team/2020-August/112610.html
** Description changed: + == Justification == + All the sub test cases in rtnetlink.sh from kselftests/net use the same variable ret to store the return value of their test result, and it will be reset to 0 in the beginning of each test. + + This will case cause false-negative result if the last case has passed + + Also, the kci_test_encap() test in rtnetlink.sh is actually composed by + two different sub-tests, kci_test_encap_vxlan() and kci_test_encap_fou() + + Therefore we should check the test result of these two to avoid false- + negative result. + + == Fixes == + * c2a4d2747996 ("selftests: rtnetlink: correct the final return value for the test") + * 72f70c159b53 ("selftests: rtnetlink: make kci_test_encap() return sub-test result") + + This issue is affecting our kernels from B to G, there is no such test + in X so it can be skipped. + + The first patch can be cherry-picked for E/F/G, but needs to be backported for B/D as they're missing some tests. + The second patch can be cherry-picked for all affected kernels. + + == Test == + Manually tested. The test will fail as expected: + $ sudo ./rtnetlink.sh + PASS: policy routing + PASS: route get + echo $? + PASS: preferred_lft addresses have expired + PASS: promote_secondaries complete + PASS: tc htb hierarchy + PASS: gre tunnel endpoint + PASS: gretap + PASS: ip6gretap + PASS: erspan + PASS: ip6erspan + PASS: bridge setup + PASS: ipv6 addrlabel + PASS: set ifalias a28e0b75-bcc7-4b62-8f5a-381215796229 for test-dummy0 + PASS: vrf + PASS: vxlan + FAIL: can't add fou port 7777, skipping test + PASS: macsec + PASS: ipsec + PASS: ipsec_offload + PASS: bridge fdb get + PASS: neigh get + $ echo $? + 1 + + == Regression Potential == + Low, changes limited to testing tools. It's expected to see this test failing after applying these patches, since it is reflecting the real test result. + + == Original Bug Report == In this test, it uses ret to store the return value of each test. However, this value will be reset to 0 in the beginning of each test. In the end of this test, it will judge PASS/FAIL base on this value. Thus this will cause false-negative in some cases. Below is an example for the test on Bionic OEM-OSP1, test "ip6erspan", "erspan", "ip6gretap" failed with return value 255, but the return value will soon be overridden with 0 if following test passed without any issue (I made the test to print === RET $ret === line for debugging purpose): PASS: policy routing === RET 0 === PASS: route get === RET 0 === PASS: preferred_lft addresses have expired === RET 0 === PASS: tc htb hierarchy === RET 0 === PASS: gre tunnel endpoint === RET 0 === PASS: gretap === RET 0 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ hoplimit TTL ] - [ encaplimit ELIM ] - [ tclass TCLASS ] - [ flowlabel FLOWLABEL ] - [ dscp inherit ] - [ fwmark MARK ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ erspan IDX ] - - Where: ADDR := IPV6_ADDRESS - TTL := { 0..255 } (default=64) - KEY := { DOTTED_QUAD | NUMBER } - ELIM := { none | 0..255 }(default=4) - TCLASS := { 0x0..0xff | inherit } - FLOWLABEL := { 0x0..0xfffff | inherit } - MARK := { 0x0..0xffffffff | inherit } + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ hoplimit TTL ] + [ encaplimit ELIM ] + [ tclass TCLASS ] + [ flowlabel FLOWLABEL ] + [ dscp inherit ] + [ fwmark MARK ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ erspan IDX ] + + Where: ADDR := IPV6_ADDRESS + TTL := { 0..255 } (default=64) + KEY := { DOTTED_QUAD | NUMBER } + ELIM := { none | 0..255 }(default=4) + TCLASS := { 0x0..0xff | inherit } + FLOWLABEL := { 0x0..0xfffff | inherit } + MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6gretap00" FAIL: ip6gretap === RET 255 === Usage: ... { gre | gretap | erspan } [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ ttl TTL ] - [ tos TOS ] - [ [no]pmtudisc ] - [ [no]ignore-df ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ external ] - [ fwmark MARK ] - [ erspan IDX ] + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ ttl TTL ] + [ tos TOS ] + [ [no]pmtudisc ] + [ [no]ignore-df ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ external ] + [ fwmark MARK ] + [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } - TOS := { NUMBER | inherit } - TTL := { 1..255 | inherit } - KEY := { DOTTED_QUAD | NUMBER } - MARK := { 0x0..0xffffffff } + TOS := { NUMBER | inherit } + TTL := { 1..255 | inherit } + KEY := { DOTTED_QUAD | NUMBER } + MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" Usage: ... { gre | gretap | erspan } [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ ttl TTL ] - [ tos TOS ] - [ [no]pmtudisc ] - [ [no]ignore-df ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ external ] - [ fwmark MARK ] - [ erspan IDX ] + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ ttl TTL ] + [ tos TOS ] + [ [no]pmtudisc ] + [ [no]ignore-df ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ external ] + [ fwmark MARK ] + [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } - TOS := { NUMBER | inherit } - TTL := { 1..255 | inherit } - KEY := { DOTTED_QUAD | NUMBER } - MARK := { 0x0..0xffffffff } + TOS := { NUMBER | inherit } + TTL := { 1..255 | inherit } + KEY := { DOTTED_QUAD | NUMBER } + MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" FAIL: erspan === RET 255 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ hoplimit TTL ] - [ encaplimit ELIM ] - [ tclass TCLASS ] - [ flowlabel FLOWLABEL ] - [ dscp inherit ] - [ fwmark MARK ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ erspan IDX ] - - Where: ADDR := IPV6_ADDRESS - TTL := { 0..255 } (default=64) - KEY := { DOTTED_QUAD | NUMBER } - ELIM := { none | 0..255 }(default=4) - TCLASS := { 0x0..0xff | inherit } - FLOWLABEL := { 0x0..0xfffff | inherit } - MARK := { 0x0..0xffffffff | inherit } - Cannot find device "ip6erspan00" - Cannot find device "ip6erspan00" - Cannot find device "ip6erspan00" - Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ hoplimit TTL ] - [ encaplimit ELIM ] - [ tclass TCLASS ] - [ flowlabel FLOWLABEL ] - [ dscp inherit ] - [ fwmark MARK ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ erspan IDX ] - - Where: ADDR := IPV6_ADDRESS - TTL := { 0..255 } (default=64) - KEY := { DOTTED_QUAD | NUMBER } - ELIM := { none | 0..255 }(default=4) - TCLASS := { 0x0..0xff | inherit } - FLOWLABEL := { 0x0..0xfffff | inherit } - MARK := { 0x0..0xffffffff | inherit } - Cannot find device "ip6erspan00" - Cannot find device "ip6erspan00" - Cannot find device "ip6erspan00" - Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] - [ local ADDR ] - [ [i|o]seq ] - [ [i|o]key KEY ] - [ [i|o]csum ] - [ hoplimit TTL ] - [ encaplimit ELIM ] - [ tclass TCLASS ] - [ flowlabel FLOWLABEL ] - [ dscp inherit ] - [ fwmark MARK ] - [ dev PHYS_DEV ] - [ noencap ] - [ encap { fou | gue | none } ] - [ encap-sport PORT ] - [ encap-dport PORT ] - [ [no]encap-csum ] - [ [no]encap-csum6 ] - [ [no]encap-remcsum ] - [ erspan IDX ] - - Where: ADDR := IPV6_ADDRESS - TTL := { 0..255 } (default=64) - KEY := { DOTTED_QUAD | NUMBER } - ELIM := { none | 0..255 }(default=4) - TCLASS := { 0x0..0xff | inherit } - FLOWLABEL := { 0x0..0xfffff | inherit } - MARK := { 0x0..0xffffffff | inherit } + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ hoplimit TTL ] + [ encaplimit ELIM ] + [ tclass TCLASS ] + [ flowlabel FLOWLABEL ] + [ dscp inherit ] + [ fwmark MARK ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ erspan IDX ] + + Where: ADDR := IPV6_ADDRESS + TTL := { 0..255 } (default=64) + KEY := { DOTTED_QUAD | NUMBER } + ELIM := { none | 0..255 }(default=4) + TCLASS := { 0x0..0xff | inherit } + FLOWLABEL := { 0x0..0xfffff | inherit } + MARK := { 0x0..0xffffffff | inherit } + Cannot find device "ip6erspan00" + Cannot find device "ip6erspan00" + Cannot find device "ip6erspan00" + Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ hoplimit TTL ] + [ encaplimit ELIM ] + [ tclass TCLASS ] + [ flowlabel FLOWLABEL ] + [ dscp inherit ] + [ fwmark MARK ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ erspan IDX ] + + Where: ADDR := IPV6_ADDRESS + TTL := { 0..255 } (default=64) + KEY := { DOTTED_QUAD | NUMBER } + ELIM := { none | 0..255 }(default=4) + TCLASS := { 0x0..0xff | inherit } + FLOWLABEL := { 0x0..0xfffff | inherit } + MARK := { 0x0..0xffffffff | inherit } + Cannot find device "ip6erspan00" + Cannot find device "ip6erspan00" + Cannot find device "ip6erspan00" + Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] + [ local ADDR ] + [ [i|o]seq ] + [ [i|o]key KEY ] + [ [i|o]csum ] + [ hoplimit TTL ] + [ encaplimit ELIM ] + [ tclass TCLASS ] + [ flowlabel FLOWLABEL ] + [ dscp inherit ] + [ fwmark MARK ] + [ dev PHYS_DEV ] + [ noencap ] + [ encap { fou | gue | none } ] + [ encap-sport PORT ] + [ encap-dport PORT ] + [ [no]encap-csum ] + [ [no]encap-csum6 ] + [ [no]encap-remcsum ] + [ erspan IDX ] + + Where: ADDR := IPV6_ADDRESS + TTL := { 0..255 } (default=64) + KEY := { DOTTED_QUAD | NUMBER } + ELIM := { none | 0..255 }(default=4) + TCLASS := { 0x0..0xff | inherit } + FLOWLABEL := { 0x0..0xfffff | inherit } + MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" FAIL: ip6erspan === RET 255 === PASS: bridge setup === RET 0 === PASS: ipv6 addrlabel === RET 0 === PASS: set ifalias a1214e60-3ac4-4fd4-8a98-aac4c0b4bfab for test-dummy0 === RET 0 === PASS: vrf === RET 0 === PASS: vxlan FAIL: can't add fou port 7777, skipping test === RET 0 === PASS: macsec === RET 0 === PASS: ipsec === RET 0 === PASS: ipsec_offload === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === === RET 0 === ** Description changed: == Justification == - All the sub test cases in rtnetlink.sh from kselftests/net use the same variable ret to store the return value of their test result, and it will be reset to 0 in the beginning of each test. - - This will case cause false-negative result if the last case has passed + All the sub test cases in rtnetlink.sh from kselftests/net were using + the same variable "ret" to store the return value of their test result, + and it will be reset to 0 in the beginning of each test. + + This will cause false-negative result if the last case has passed. Also, the kci_test_encap() test in rtnetlink.sh is actually composed by two different sub-tests, kci_test_encap_vxlan() and kci_test_encap_fou() - Therefore we should check the test result of these two to avoid false- - negative result. + Therefore we should check the test result of these two to avoid + false-negative result for this test case. == Fixes == * c2a4d2747996 ("selftests: rtnetlink: correct the final return value for the test") * 72f70c159b53 ("selftests: rtnetlink: make kci_test_encap() return sub-test result") This issue is affecting our kernels from B to G, there is no such test - in X so it can be skipped. - - The first patch can be cherry-picked for E/F/G, but needs to be backported for B/D as they're missing some tests. + in X thus it can be skipped. + + The first patch can be cherry-picked for E/F/G, but needs to be + backported for B/D as they're missing some tests. The second patch can be cherry-picked for all affected kernels. == Test == Manually tested. The test will fail as expected: - $ sudo ./rtnetlink.sh + $ sudo ./rtnetlink.sh PASS: policy routing PASS: route get echo $? PASS: preferred_lft addresses have expired PASS: promote_secondaries complete PASS: tc htb hierarchy PASS: gre tunnel endpoint PASS: gretap PASS: ip6gretap PASS: erspan PASS: ip6erspan PASS: bridge setup PASS: ipv6 addrlabel PASS: set ifalias a28e0b75-bcc7-4b62-8f5a-381215796229 for test-dummy0 PASS: vrf PASS: vxlan FAIL: can't add fou port 7777, skipping test PASS: macsec PASS: ipsec PASS: ipsec_offload PASS: bridge fdb get PASS: neigh get $ echo $? 1 == Regression Potential == - Low, changes limited to testing tools. It's expected to see this test failing after applying these patches, since it is reflecting the real test result. + Low, changes limited to testing tools. It's expected to see this test + failing after applying these patches, since it is reflecting the actual + test result. == Original Bug Report == In this test, it uses ret to store the return value of each test. However, this value will be reset to 0 in the beginning of each test. In the end of this test, it will judge PASS/FAIL base on this value. Thus this will cause false-negative in some cases. Below is an example for the test on Bionic OEM-OSP1, test "ip6erspan", "erspan", "ip6gretap" failed with return value 255, but the return value will soon be overridden with 0 if following test passed without any issue (I made the test to print === RET $ret === line for debugging purpose): PASS: policy routing === RET 0 === PASS: route get === RET 0 === PASS: preferred_lft addresses have expired === RET 0 === PASS: tc htb hierarchy === RET 0 === PASS: gre tunnel endpoint === RET 0 === PASS: gretap === RET 0 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6gretap00" FAIL: ip6gretap === RET 255 === Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" FAIL: erspan === RET 255 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" FAIL: ip6erspan === RET 255 === PASS: bridge setup === RET 0 === PASS: ipv6 addrlabel === RET 0 === PASS: set ifalias a1214e60-3ac4-4fd4-8a98-aac4c0b4bfab for test-dummy0 === RET 0 === PASS: vrf === RET 0 === PASS: vxlan FAIL: can't add fou port 7777, skipping test === RET 0 === PASS: macsec === RET 0 === PASS: ipsec === RET 0 === PASS: ipsec_offload === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === === RET 0 === ** Description changed: == Justification == All the sub test cases in rtnetlink.sh from kselftests/net were using the same variable "ret" to store the return value of their test result, and it will be reset to 0 in the beginning of each test. This will cause false-negative result if the last case has passed. Also, the kci_test_encap() test in rtnetlink.sh is actually composed by two different sub-tests, kci_test_encap_vxlan() and kci_test_encap_fou() Therefore we should check the test result of these two to avoid false-negative result for this test case. == Fixes == * c2a4d2747996 ("selftests: rtnetlink: correct the final return value for the test") * 72f70c159b53 ("selftests: rtnetlink: make kci_test_encap() return sub-test result") This issue is affecting our kernels from B to G, there is no such test in X thus it can be skipped. The first patch can be cherry-picked for E/F/G, but needs to be backported for B/D as they're missing some tests. The second patch can be cherry-picked for all affected kernels. == Test == Manually tested. The test will fail as expected: $ sudo ./rtnetlink.sh PASS: policy routing PASS: route get echo $? PASS: preferred_lft addresses have expired PASS: promote_secondaries complete PASS: tc htb hierarchy PASS: gre tunnel endpoint PASS: gretap PASS: ip6gretap PASS: erspan PASS: ip6erspan PASS: bridge setup PASS: ipv6 addrlabel PASS: set ifalias a28e0b75-bcc7-4b62-8f5a-381215796229 for test-dummy0 PASS: vrf PASS: vxlan FAIL: can't add fou port 7777, skipping test PASS: macsec PASS: ipsec PASS: ipsec_offload PASS: bridge fdb get PASS: neigh get $ echo $? 1 == Regression Potential == Low, changes limited to testing tools. It's expected to see this test failing after applying these patches, since it is reflecting the actual test result. + == Original Bug Report == In this test, it uses ret to store the return value of each test. However, this value will be reset to 0 in the beginning of each test. In the end of this test, it will judge PASS/FAIL base on this value. Thus this will cause false-negative in some cases. Below is an example for the test on Bionic OEM-OSP1, test "ip6erspan", "erspan", "ip6gretap" failed with return value 255, but the return value will soon be overridden with 0 if following test passed without any issue (I made the test to print === RET $ret === line for debugging purpose): PASS: policy routing === RET 0 === PASS: route get === RET 0 === PASS: preferred_lft addresses have expired === RET 0 === PASS: tc htb hierarchy === RET 0 === PASS: gre tunnel endpoint === RET 0 === PASS: gretap === RET 0 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6gretap00" FAIL: ip6gretap === RET 255 === Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" FAIL: erspan === RET 255 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" FAIL: ip6erspan === RET 255 === PASS: bridge setup === RET 0 === PASS: ipv6 addrlabel === RET 0 === PASS: set ifalias a1214e60-3ac4-4fd4-8a98-aac4c0b4bfab for test-dummy0 === RET 0 === PASS: vrf === RET 0 === PASS: vxlan FAIL: can't add fou port 7777, skipping test === RET 0 === PASS: macsec === RET 0 === PASS: ipsec === RET 0 === PASS: ipsec_offload === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === === RET 0 === ** Summary changed: - Return value in rtnetlink.sh from kselftests/net will be override in the last test + Fix false-negative return value for rtnetlink.sh in kselftests/net ** Changed in: linux (Ubuntu Bionic) Assignee: (unassigned) => Po-Hsu Lin (cypressyew) ** Changed in: linux (Ubuntu Eoan) Assignee: (unassigned) => Po-Hsu Lin (cypressyew) ** Changed in: linux (Ubuntu Focal) Assignee: (unassigned) => Po-Hsu Lin (cypressyew) ** Changed in: linux (Ubuntu Groovy) Assignee: (unassigned) => Po-Hsu Lin (cypressyew) ** Changed in: linux (Ubuntu Groovy) Status: Incomplete => In Progress ** Changed in: linux (Ubuntu Eoan) Status: New => In Progress ** Changed in: linux (Ubuntu Bionic) Status: New => In Progress ** Changed in: linux (Ubuntu Focal) Status: New => In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1890136 Title: Fix false-negative return value for rtnetlink.sh in kselftests/net Status in ubuntu-kernel-tests: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Status in linux source package in Eoan: In Progress Status in linux source package in Focal: In Progress Status in linux source package in Groovy: In Progress Bug description: == Justification == All the sub test cases in rtnetlink.sh from kselftests/net were using the same variable "ret" to store the return value of their test result, and it will be reset to 0 in the beginning of each test. This will cause false-negative result if the last case has passed. Also, the kci_test_encap() test in rtnetlink.sh is actually composed by two different sub-tests, kci_test_encap_vxlan() and kci_test_encap_fou() Therefore we should check the test result of these two to avoid false-negative result for this test case. == Fixes == * c2a4d2747996 ("selftests: rtnetlink: correct the final return value for the test") * 72f70c159b53 ("selftests: rtnetlink: make kci_test_encap() return sub-test result") This issue is affecting our kernels from B to G, there is no such test in X thus it can be skipped. The first patch can be cherry-picked for E/F/G, but needs to be backported for B/D as they're missing some tests. The second patch can be cherry-picked for all affected kernels. == Test == Manually tested. The test will fail as expected: $ sudo ./rtnetlink.sh PASS: policy routing PASS: route get echo $? PASS: preferred_lft addresses have expired PASS: promote_secondaries complete PASS: tc htb hierarchy PASS: gre tunnel endpoint PASS: gretap PASS: ip6gretap PASS: erspan PASS: ip6erspan PASS: bridge setup PASS: ipv6 addrlabel PASS: set ifalias a28e0b75-bcc7-4b62-8f5a-381215796229 for test-dummy0 PASS: vrf PASS: vxlan FAIL: can't add fou port 7777, skipping test PASS: macsec PASS: ipsec PASS: ipsec_offload PASS: bridge fdb get PASS: neigh get $ echo $? 1 == Regression Potential == Low, changes limited to testing tools. It's expected to see this test failing after applying these patches, since it is reflecting the actual test result. == Original Bug Report == In this test, it uses ret to store the return value of each test. However, this value will be reset to 0 in the beginning of each test. In the end of this test, it will judge PASS/FAIL base on this value. Thus this will cause false-negative in some cases. Below is an example for the test on Bionic OEM-OSP1, test "ip6erspan", "erspan", "ip6gretap" failed with return value 255, but the return value will soon be overridden with 0 if following test passed without any issue (I made the test to print === RET $ret === line for debugging purpose): PASS: policy routing === RET 0 === PASS: route get === RET 0 === PASS: preferred_lft addresses have expired === RET 0 === PASS: tc htb hierarchy === RET 0 === PASS: gre tunnel endpoint === RET 0 === PASS: gretap === RET 0 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6gretap00" FAIL: ip6gretap === RET 255 === Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" Usage: ... { gre | gretap | erspan } [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ ttl TTL ] [ tos TOS ] [ [no]pmtudisc ] [ [no]ignore-df ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ external ] [ fwmark MARK ] [ erspan IDX ] Where: ADDR := { IP_ADDRESS | any } TOS := { NUMBER | inherit } TTL := { 1..255 | inherit } KEY := { DOTTED_QUAD | NUMBER } MARK := { 0x0..0xffffffff } Cannot find device "erspan00" Cannot find device "erspan00" Cannot find device "erspan00" FAIL: erspan === RET 255 === Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Cannot find device "ip6erspan00" Usage: ... { ip6gre | ip6gretap | ip6erspan} [ remote ADDR ] [ local ADDR ] [ [i|o]seq ] [ [i|o]key KEY ] [ [i|o]csum ] [ hoplimit TTL ] [ encaplimit ELIM ] [ tclass TCLASS ] [ flowlabel FLOWLABEL ] [ dscp inherit ] [ fwmark MARK ] [ dev PHYS_DEV ] [ noencap ] [ encap { fou | gue | none } ] [ encap-sport PORT ] [ encap-dport PORT ] [ [no]encap-csum ] [ [no]encap-csum6 ] [ [no]encap-remcsum ] [ erspan IDX ] Where: ADDR := IPV6_ADDRESS TTL := { 0..255 } (default=64) KEY := { DOTTED_QUAD | NUMBER } ELIM := { none | 0..255 }(default=4) TCLASS := { 0x0..0xff | inherit } FLOWLABEL := { 0x0..0xfffff | inherit } MARK := { 0x0..0xffffffff | inherit } Cannot find device "ip6erspan00" FAIL: ip6erspan === RET 255 === PASS: bridge setup === RET 0 === PASS: ipv6 addrlabel === RET 0 === PASS: set ifalias a1214e60-3ac4-4fd4-8a98-aac4c0b4bfab for test-dummy0 === RET 0 === PASS: vrf === RET 0 === PASS: vxlan FAIL: can't add fou port 7777, skipping test === RET 0 === PASS: macsec === RET 0 === PASS: ipsec === RET 0 === PASS: ipsec_offload === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === SKIP: fdb get tests: iproute2 too old === RET 0 === === RET 0 === To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1890136/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp