On Sat, 13 Mar 2021 16:13:54 -0600 Bruce Dubbs <bruce.du...@gmail.com> wrote:
> On 3/12/21 5:48 PM, Bruce Dubbs wrote: > > On 3/12/21 5:25 PM, Glenn Washburn wrote: > >> Hi Bruce, > >> > >> On Fri, 12 Mar 2021 15:30:01 -0600 > >> Bruce Dubbs <bruce.du...@gmail.com> wrote: > >> > >>> On 3/12/21 1:57 PM, Daniel Kiper wrote: > > [snip] > > >>> For LFS, we do a very simple build in a sparse environment. Here > >>> are a few observations. > >>> > >>> We use: > >>> > >>> ./configure --prefix=/usr \ > >>> --sbindir=/sbin \ > >>> --sysconfdir=/etc \ > >>> --disable-efiemu \ > >>> --disable-werror > > [snip] > > >> I'm curious, what compiler and version are you using? > > > > $ gcc --version > > gcc (GCC) 10.2.0 That might be why I haven't caught it then. The GitLab CI is using at most 10.1.0, until the kernel.org prebuilt binaries get added for more recent versions. > > $ bison --version > > bison (GNU Bison) 3.7.5 > > > >>> For make check, I get 43 failures. Many appear to be due to our > >>> sparse environment. In most cases I think these tests should be > >>> SKIP instead of FAIL, but I'll follow up with a more detailed > >>> report later. > >> > >> I'm looking forward to the report. I think ideally the tests > >> should use a very minimal common set of utilities (with some > >> obvious exceptions like qemu and mkfs.*). The tests should only > >> SKIP if its not an appropriate test for the configured target. > >> More likely, they should ERROR to indicated that there was a > >> failure to run the test, not a failure of what the test was > >> testing. This lets the tester know that there's likely an > >> environmental issue that needs to be fixed to get tests working as > >> they should. > > I'll note that several tests require being run as the root user. I > cannot find where that is specified. I only figured that out by > directly looking at the test scripts. I don't think running the test suite is documented anywhere, and it should be (likely in the developer manual). Is there some documentation that I'm not aware of? > When I build in a full LFS/BLFS system I get: > > # TOTAL: 82 > # PASS: 67 > # SKIP: 11 > # XFAIL: 0 > # FAIL: 4 > # XPASS: 0 > # ERROR: 0 > > The SKIPs are due to not having the expected executables. > squashfs_test > hfsplus_test > minixfs_test > f2fs_test > nilfs2_test > romfs_test > exfat_test > udf_test > hfs_test > zfs_test Yes, and they should be ERRORs so that folks testing understand that there's an unexpected issue. > lzocompress_test > This test depends on an lzop executable which in not a part of the > xz-5.2.5 distribution. I found it is from a lzop source package > which is new to us at LFS. > > I doubt many systems will have most of the the software needed for > these tests, but perhaps a listing of what GRUB expects would be > useful. The GitLab integration config that I created has a list of ubuntu/debian packages that I needed to get these tests running. And due to some limitations of GitLab shared runners there are a few tests that don't run (like zfs). The list of packages start at this line in the config: https://gitlab.com/gnu-grub/grub/-/blob/a971d40f5d31b8f9ddec3eabfac1c232ca1cf240/.gitlab-ci.yml#L457 > The FAILing tests are a bit more troubling: > reiserfs_test > mount: /tmp/tmp.t6gBOmoH4g/reiserfs_old_rw: can't read superblock > on /dev/loop0. > MOUNT FAILED. > > I have reiserfsprogs-3.6.27 installed. Its not easy to instrument the tests currently to provide shell tracing, which would help here. Though I have some patches in the pipeline that will help in this regard. Do you have loopback devices working correctly and available? > ---- > fat_test > mkfs.vfat: Labels with characters *?.,;:/\|+=<>[]" are not allowed > mkfs.fat 4.2 (2021-01-31) > FAIL fat_test (exit status: 1) > > I have dosfstools-4.2 installed. I wonder if this is due to a newer version dosfstools? The GitLab CI was setup to test on Ubuntu xenial which is using dosfstools version 3.0.28 and is not hitting this issue. I did this to help make sure we're maintaining backward compatibility with older systems. > ---- > btrfs_test > ERROR: invalid sectorsize 256, expected range is [4K, 64K] > btrfs-progs v5.10.1 > See http://btrfs.wiki.kernel.org for more information. > FAIL btrfs_test (exit status: 1) > > I have btrfs-progs-v5.10.1 installed. Again, haven't seen this either and perhaps due to a newer version. GitLab CI is using version 4.4. > ---- > grub_func_test > FAIL grub_func_test (exit status: 1) > > I cannot determine from the test what is expected. > ---- grub_func_test is a suite of functional tests. All tests should pass. Currently several do not, and it is known that the tests need to be updated. So some of these tests are not valid. The GitLab CI currently ignores the result of videotest_checksum, gfxterm_menu, and cmdline_cat_test [1]. > > It would appear that many of the GRUB regression test results are > distro specific and not really GRUB tests. At a minimum, I don't think any of these indicate issues in the GRUB code. But I think they indicate that some tests will need to be upgraded in the future to account for changes in test dependencies. Glenn [1] https://gitlab.com/gnu-grub/grub/-/blob/a971d40f5d31b8f9ddec3eabfac1c232ca1cf240/.gitlab-ci.yml#L160 _______________________________________________ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
