[Touch-packages] [Bug 2019023] Re: Fix shell test suite
This bug was fixed in the package iptables - 1.8.4-3ubuntu2.1 --- iptables (1.8.4-3ubuntu2.1) focal; urgency=medium [ Louis Bouchard ] * d/p/0001-libiptc-Fix-for-segfault-when-renaming-a-chain.patch: apply upstream fix for segfault when renaming a chain. Includes a test case (LP: #1992454) [ Andreas Hasenack ] * Fix shell tests, and run them as DEP8 (LP: #2019023): - d/p/0501-fix-firewalld-shell-test.patch: fix the globbing expression which was used to decide whether to sort the restore output or not before comparing it to the good case - d/t/control: run all shell tests as DEP8 -- Andreas Hasenack Tue, 09 May 2023 15:39:57 -0300 ** Changed in: iptables (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to iptables in Ubuntu. https://bugs.launchpad.net/bugs/2019023 Title: Fix shell test suite Status in iptables package in Ubuntu: Fix Released Status in iptables source package in Focal: Fix Released Bug description: [ Impact ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests in focal only: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) The upstream fix includes other similar changes in other tests, but in the case of focal, the above is the minimal fix needed. Note that this shell test suite is not being run in focal, just in later ubuntu releases. But since the fix for #1992454 is adding such a test, I decided to fix the shell test run and add it to the existing DEP8 tests for focal via this bug, so we have test parity between focal and later ubuntu releases. 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 [ Test Plan ] Verify that the DEP8 tests now include a run-tests.sh test suite, and that it passes. [ Where problems could occur ] If the fix is incorrect, it would affect only the already-failing firewalld test. But in addition to fixing that test, we are now also including a full test run of all shell tests, something which wasn't being done for focal until now. While these tests are passing now, they could fail in a future iptables SRU, or turn out to be flaky. They are being run in ubuntu releases after focal, though, so that is a good sign. [ Other Info ] This fix is being included in the same upload as bug #1992454. [ Original Description ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/iptables/+bug/2019023/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2019023] Re: Fix shell test suite
Focal amd64 dep8 run: https://autopkgtest.ubuntu.com/results/autopkgtest- focal/focal/amd64/i/iptables/20230516_045413_de974@/log.gz It now contains a new dep8 test, labeled command17, which runs run- tests.sh: autopkgtest [04:53:52]: test command17: chmod +x ./iptables/tests/shell/testcases/chain/0006rename-segfault_0; cd iptables/tests/shell; ./run-tests.sh --host Which runs the shell test suite, including the test for bug #1992454: autopkgtest [04:53:52]: test command17: chmod +x ./iptables/tests/shell/testcases/chain/0006rename-segfault_0; cd iptables/tests/shell; ./run-tests.sh --host autopkgtest [04:53:52]: test command17: [--- I: [EXECUTING] ././testcases/arptables/0001-arptables-save-restore_0 [1A[KI: [OK] ././testcases/arptables/0001-arptables-save-restore_0 (...) I: [EXECUTING] ././testcases/chain/0006rename-segfault_0 [1A[KI: [OK] ././testcases/chain/0006rename-segfault_0 (...) And this suite passes: (...) I: [EXECUTING] ././testcases/nft-only/0003delete-with-comment_0 [1A[KI: [OK] ././testcases/nft-only/0003delete-with-comment_0 I: nft results: [OK] 41 [FAILED] 0 [TOTAL] 41 I: combined results: [OK] 82 [FAILED] 0 [TOTAL] 82 autopkgtest [04:54:00]: test command17: ---] autopkgtest [04:54:00]: test command17: - - - - - - - - - - results - - - - - - - - - - command17PASS Focal verification succeeded. ** Tags removed: verification-needed-focal ** Tags added: verification-done-focal -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to iptables in Ubuntu. https://bugs.launchpad.net/bugs/2019023 Title: Fix shell test suite Status in iptables package in Ubuntu: Fix Released Status in iptables source package in Focal: Fix Committed Bug description: [ Impact ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests in focal only: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) The upstream fix includes other similar changes in other tests, but in the case of focal, the above is the minimal fix needed. Note that this shell test suite is not being run in focal, just in later ubuntu releases. But since the fix for #1992454 is adding such a test, I decided to fix the shell test run and add it to the existing DEP8 tests for focal via this bug, so we have test parity between focal and later ubuntu releases. 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 [ Test Plan ] Verify that the DEP8 tests now include a run-tests.sh test suite, and that it passes. [ Where problems could occur ] If the fix is incorrect, it would affect only the already-failing firewalld test. But in addition to fixing that test, we are now also including a full test run of all shell tests, something which wasn't being done for focal until now. While these tests are passing now, they could fail in a future iptables SRU, or turn out to be flaky. They are being run in ubuntu releases after focal, though, so that is a good sign. [ Other Info ] This fix is being included in the same upload as bug #1992454. [ Original Description ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/iptables/+bug/2019023/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help :
[Touch-packages] [Bug 2019023] Re: Fix shell test suite
Hello Andreas, or anyone else affected, Accepted iptables into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/iptables/1.8.4-3ubuntu2.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-focal. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: iptables (Ubuntu Focal) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to iptables in Ubuntu. https://bugs.launchpad.net/bugs/2019023 Title: Fix shell test suite Status in iptables package in Ubuntu: Fix Released Status in iptables source package in Focal: Fix Committed Bug description: [ Impact ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests in focal only: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) The upstream fix includes other similar changes in other tests, but in the case of focal, the above is the minimal fix needed. Note that this shell test suite is not being run in focal, just in later ubuntu releases. But since the fix for #1992454 is adding such a test, I decided to fix the shell test run and add it to the existing DEP8 tests for focal via this bug, so we have test parity between focal and later ubuntu releases. 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 [ Test Plan ] Verify that the DEP8 tests now include a run-tests.sh test suite, and that it passes. [ Where problems could occur ] If the fix is incorrect, it would affect only the already-failing firewalld test. But in addition to fixing that test, we are now also including a full test run of all shell tests, something which wasn't being done for focal until now. While these tests are passing now, they could fail in a future iptables SRU, or turn out to be flaky. They are being run in ubuntu releases after focal, though, so that is a good sign. [ Other Info ] This fix is being included in the same upload as bug #1992454. [ Original Description ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/iptables/+bug/2019023/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 2019023] Re: Fix shell test suite
** Description changed: [ Impact ] - * An explanation of the effects of the bug on users and + The shell test suite (iptables/tests/shell/run-tests.sh) is currently + failing on the firewalld tests in focal only: - * justification for backporting the fix to the stable release. + W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 + W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 - * In addition, it is helpful, but not required, to include an -explanation of how the upload fixes this bug. + After some troubleshooting, it turns out this is happening because of an + unsorted order in the output of iptables-save, which was fixed[1] in + later releases of iptables. The code was trying to compensate for that, + but there was a small mistake[2] in a case/esac globbing: + + case "$XT_MULTI" in + -*/xtables-nft-multi) + +*xtables-nft-multi) + + The upstream fix includes other similar changes in other tests, but in + the case of focal, the above is the minimal fix needed. + + Note that this shell test suite is not being run in focal, just in later + ubuntu releases. But since the fix for #1992454 is adding such a test, I + decided to fix the shell test run and add it to the existing DEP8 tests + for focal via this bug, so we have test parity between focal and later + ubuntu releases. + + + 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef + 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 + [ Test Plan ] + Verify that the DEP8 tests now include a run-tests.sh test suite, and that it passes. - * detailed instructions how to reproduce the bug - - * these should allow someone who is not familiar with the affected -package to reproduce the bug and verify that the updated package fixes -the problem. - - * if other testing is appropriate to perform before landing this update, -this should also be described here. [ Where problems could occur ] - - * Think about what the upload changes in the software. Imagine the change is -wrong or breaks something else: how would this show up? - - * It is assumed that any SRU candidate patch is well-tested before -upload and has a low overall risk of regression, but it's important -to make the effort to think about what ''could'' happen in the -event of a regression. - - * This must '''never''' be "None" or "Low", or entirely an argument as to why -your upload is low risk. - - * This both shows the SRU team that the risks have been considered, -and provides guidance to testers in regression-testing the SRU. + If the fix is incorrect, it would affect only the already-failing firewalld test. But in addition to fixing that test, we are now also including a full test run of all shell tests, something which wasn't being done for focal until now. While these tests are passing now, they could fail in a future iptables SRU, or turn out to be flaky. They are being run in ubuntu releases after focal, though, so that is a good sign. [ Other Info ] - - * Anything else you think is useful to include - * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board - * and address these questions in advance + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance [ Original Description ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 ** Description changed: [ Impact ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests in focal only: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for
[Touch-packages] [Bug 2019023] Re: Fix shell test suite
** Description changed: + [ Impact ] + + * An explanation of the effects of the bug on users and + + * justification for backporting the fix to the stable release. + + * In addition, it is helpful, but not required, to include an +explanation of how the upload fixes this bug. + + [ Test Plan ] + + * detailed instructions how to reproduce the bug + + * these should allow someone who is not familiar with the affected +package to reproduce the bug and verify that the updated package fixes +the problem. + + * if other testing is appropriate to perform before landing this update, +this should also be described here. + + [ Where problems could occur ] + + * Think about what the upload changes in the software. Imagine the change is +wrong or breaks something else: how would this show up? + + * It is assumed that any SRU candidate patch is well-tested before +upload and has a low overall risk of regression, but it's important +to make the effort to think about what ''could'' happen in the +event of a regression. + + * This must '''never''' be "None" or "Low", or entirely an argument as to why +your upload is low risk. + + * This both shows the SRU team that the risks have been considered, +and provides guidance to testers in regression-testing the SRU. + + [ Other Info ] + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance + + [ Original Description ] + The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of an unsorted order in the output of iptables-save, which was fixed[1] in later releases of iptables. The code was trying to compensate for that, but there was a small mistake[2] in a case/esac globbing: case "$XT_MULTI" in -*/xtables-nft-multi) +*xtables-nft-multi) - 1. https://git.netfilter.org/iptables/commit/?id=e28cf12cf50b9e2e0114f04331635fc122cb8aef 2. https://git.netfilter.org/iptables/commit/?id=2b2b7948c1960ba4680677664ff58477be869de6 ** Also affects: iptables (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: iptables (Ubuntu Focal) Status: New => In Progress ** Changed in: iptables (Ubuntu Focal) Assignee: (unassigned) => Andreas Hasenack (ahasenack) ** Changed in: iptables (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to iptables in Ubuntu. https://bugs.launchpad.net/bugs/2019023 Title: Fix shell test suite Status in iptables package in Ubuntu: Fix Released Status in iptables source package in Focal: In Progress Bug description: [ Impact ] * An explanation of the effects of the bug on users and * justification for backporting the fix to the stable release. * In addition, it is helpful, but not required, to include an explanation of how the upload fixes this bug. [ Test Plan ] * detailed instructions how to reproduce the bug * these should allow someone who is not familiar with the affected package to reproduce the bug and verify that the updated package fixes the problem. * if other testing is appropriate to perform before landing this update, this should also be described here. [ Where problems could occur ] * Think about what the upload changes in the software. Imagine the change is wrong or breaks something else: how would this show up? * It is assumed that any SRU candidate patch is well-tested before upload and has a low overall risk of regression, but it's important to make the effort to think about what ''could'' happen in the event of a regression. * This must '''never''' be "None" or "Low", or entirely an argument as to why your upload is low risk. * This both shows the SRU team that the risks have been considered, and provides guidance to testers in regression-testing the SRU. [ Other Info ] * Anything else you think is useful to include * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board * and address these questions in advance [ Original Description ] The shell test suite (iptables/tests/shell/run-tests.sh) is currently failing on the firewalld tests: W: [FAILED] ././testcases/firewalld-restore/0001-firewalld_0: expected 0 but got 1 W: [FAILED] ././testcases/firewalld-restore/0002-firewalld-restart_0: expected 0 but got 1 After some troubleshooting, it turns out this is happening because of
