[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Nathan Whitehorn changed: What|Removed |Added Resolution|--- |FIXED Status|New |Closed -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #15 from commit-h...@freebsd.org --- A commit in branch releng/13.0 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3ef46634cd5a6ba0e156d5da095bbee5f63ec1cd commit 3ef46634cd5a6ba0e156d5da095bbee5f63ec1cd Author: Nathan Whitehorn AuthorDate: 2021-03-23 13:19:42 + Commit: Nathan Whitehorn CommitDate: 2021-03-23 19:22:20 + Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Approved by:re (gjb) Reviewed by:gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 (cherry picked from commit c2f16c595eb51c6e0cb6ece3f6f078d738019059) (cherry picked from commit 4601382e1362352f17a33e4ed38db5dcfe3f6be5) usr.sbin/bsdinstall/partedit/partedit.c | 39 + usr.sbin/bsdinstall/scripts/script | 30 ++--- 2 files changed, 61 insertions(+), 8 deletions(-) -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #14 from commit-h...@freebsd.org --- A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=4601382e1362352f17a33e4ed38db5dcfe3f6be5 commit 4601382e1362352f17a33e4ed38db5dcfe3f6be5 Author: Nathan Whitehorn AuthorDate: 2021-03-23 13:19:42 + Commit: Nathan Whitehorn CommitDate: 2021-03-23 19:21:33 + Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Approved by:re (gjb) Reviewed by:gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 (cherry picked from commit c2f16c595eb51c6e0cb6ece3f6f078d738019059) usr.sbin/bsdinstall/partedit/partedit.c | 39 + usr.sbin/bsdinstall/scripts/script | 30 ++--- 2 files changed, 61 insertions(+), 8 deletions(-) -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #13 from commit-h...@freebsd.org --- A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c2f16c595eb51c6e0cb6ece3f6f078d738019059 commit c2f16c595eb51c6e0cb6ece3f6f078d738019059 Author: Nathan Whitehorn AuthorDate: 2021-03-23 13:19:42 + Commit: Nathan Whitehorn CommitDate: 2021-03-23 13:29:54 + Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Reviewed by:gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 usr.sbin/bsdinstall/partedit/partedit.c | 39 + usr.sbin/bsdinstall/scripts/script | 30 ++--- 2 files changed, 61 insertions(+), 8 deletions(-) -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #12 from Nathan Whitehorn --- Related things: https://reviews.freebsd.org/D29380 <- patch in review here https://github.com/libarchive/libarchive/issues/1516 <- libarchive issue here -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Bug 254395] bsdinstall: fail script install after BETA3
On Fri, Mar 19, 2021 at 5:45 PM Chris wrote: > On 2021-03-19 15:55, Warner Losh wrote: > > On Fri, Mar 19, 2021 at 4:49 PM Toomas Soome via freebsd-current < > > freebsd-current@freebsd.org> wrote: > > > >> > >> > >> > On 20. Mar 2021, at 00:21, Yuri Pankov wrote: > >> > > >> > Chris wrote: > >> >> On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: > >> >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 > >> >>> > >> >>> Nathan Whitehorn changed: > >> >>> > >> >>>What|Removed |Added > >> >>> > >> > > >> >>> > >> >>>Severity|Affects Only Me |Affects Some People > >> >>> CC||i...@freebsd.org > >> >>>Priority|--- |Normal > >> >>> > >> >>> --- Comment #6 from Nathan Whitehorn --- > >> >>> Thanks for the suggestion about the documentation -- I've updated > the > >> >>> man page. > >> >>> > >> >>> The core problem here is that our tar can't extract archives to > FAT32 > >> >>> with > >> >>> default options, since it treats inability to set modification time > as > >> >>> a fatal > >> >>> error and FAT32 doesn't let you do that on the root directory. As > >> >>> such, any > >> >>> file in the release tarballs can't be extracted to FAT32. For > >> interactive > >> >>> installations, the bsdinstall distextract tool, a CURSES-y frontend > to > >> >>> libarchive, solves this by ignoring ctime/mtime errors. > >> >>> > >> >>> Some extra commentary on solutions, so it can be in one place. > >> >>> Possibilities > >> >>> are: > >> >>> > >> >>> 1. We drop /boot/efi from mtree. That will result in it not > existing in > >> >>> base.txz, solving this issue, but will result in it not being in > >> >>> mtree. It will > >> >>> also leave in place an identical bug that will break scripted > >> >>> installation on > >> >>> bare-metal POWER8 and POWER9 systems, although that is a tier-2 > >> platform. > >> >>> > >> >>> 2. We add an option to tar to ignore failure in setting ctime/mtime, > >> >>> like the > >> >>> interactive installer uses. This has the difficulty that the patch > is > >> >>> hacky and > >> >>> would have to go through upstream. > >> >>> > >> >>> 3. We go back to using distextract for scripted installations as > well > >> as > >> >>> interactive ones, reverting > d7640440fb644fde697f62fdff0b55aa3a4d5ef7. > >> >>> This > >> >>> fixes this issue but will result in installation failures for > scripted > >> >>> installs > >> >>> without a controlling tty. (It will also add nice progress bars to > >> >>> scripted > >> >>> installs). > >> >>> > >> >>> 4. We do --exclude /boot/efi when running tar, then mkdir -p it by > hand > >> >>> afterward. This is incredibly hacky and otherwise essentially > >> >>> functionally > >> >>> equivalent to #1. Like #1, it will fix this issue and has no obvious > >> >>> functional > >> >>> downside, but leaves scripted installs bare-metal POWER8 and POWER9 > >> >>> broken. > >> >>> > >> >>> 5. We patch the file system driver to (pretend to) allow setting > times > >> >>> on the > >> >>> mount point. I don't want to do this, since I don't want to solve > this > >> >>> in the > >> >>> kernel at RC3 and I don't like it pretending to do things it can't > do. > >> >> > >> >> 6. (my favorite) do NOT require that the efi/ partition be in > strictly > >> a > >> >> fat32 format. I mean fat32 is not strictly required as the format > for > >> >> the efi > >> >> partition. It is simply _assumed_ to be the required format and as > >> >> such, the > >> >> one used in so many cases. > >> > > >> > Wrong, see "13.3 File System Format" in UEFI specification. > >> > > >> > >> it is not as simple as that:) > >> > >> > >> 13.3.1.1 is more specific: > >> = > >> The EFI firmware must support the FAT32, FAT16, and FAT12 variants of > the > >> EFI file system. What variant of EFI FAT to use is defined by the size > of > >> the media. The rules defining the relationship between media size and > FAT > >> variants is defined in the specification for the EFI file system. > >> > > > > We've also seen a few non-conformant systems where FAT12 support has been > > removed, so there's also a bit of real-world experience that goes along > > with reading the UEFI specification :(. > I suppose that may be part of where my understanding came from. I've > got a couple of "hackintoshes" that dual-boot OS X, and FreeBSD. Part of > the process was working with the EFI partition (ESP) to get recognition > of both OS's in the (EFI) boot menu, as well as getting the firmware > properly functional in both instances. I also draw from a long article > by a boot manager author whom insisted that fat32 was not a requirement. > I've not got the time ATM to dig up the article. But it had many pointers > to the EFI spec to prove the point. All of which I verified. > In any case,
Re: [Bug 254395] bsdinstall: fail script install after BETA3
On 2021-03-19 15:55, Warner Losh wrote: On Fri, Mar 19, 2021 at 4:49 PM Toomas Soome via freebsd-current < freebsd-current@freebsd.org> wrote: > On 20. Mar 2021, at 00:21, Yuri Pankov wrote: > > Chris wrote: >> On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 >>> >>> Nathan Whitehorn changed: >>> >>>What|Removed |Added >>> >>> >>>Severity|Affects Only Me |Affects Some People >>> CC||i...@freebsd.org >>>Priority|--- |Normal >>> >>> --- Comment #6 from Nathan Whitehorn --- >>> Thanks for the suggestion about the documentation -- I've updated the >>> man page. >>> >>> The core problem here is that our tar can't extract archives to FAT32 >>> with >>> default options, since it treats inability to set modification time as >>> a fatal >>> error and FAT32 doesn't let you do that on the root directory. As >>> such, any >>> file in the release tarballs can't be extracted to FAT32. For interactive >>> installations, the bsdinstall distextract tool, a CURSES-y frontend to >>> libarchive, solves this by ignoring ctime/mtime errors. >>> >>> Some extra commentary on solutions, so it can be in one place. >>> Possibilities >>> are: >>> >>> 1. We drop /boot/efi from mtree. That will result in it not existing in >>> base.txz, solving this issue, but will result in it not being in >>> mtree. It will >>> also leave in place an identical bug that will break scripted >>> installation on >>> bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. >>> >>> 2. We add an option to tar to ignore failure in setting ctime/mtime, >>> like the >>> interactive installer uses. This has the difficulty that the patch is >>> hacky and >>> would have to go through upstream. >>> >>> 3. We go back to using distextract for scripted installations as well as >>> interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. >>> This >>> fixes this issue but will result in installation failures for scripted >>> installs >>> without a controlling tty. (It will also add nice progress bars to >>> scripted >>> installs). >>> >>> 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand >>> afterward. This is incredibly hacky and otherwise essentially >>> functionally >>> equivalent to #1. Like #1, it will fix this issue and has no obvious >>> functional >>> downside, but leaves scripted installs bare-metal POWER8 and POWER9 >>> broken. >>> >>> 5. We patch the file system driver to (pretend to) allow setting times >>> on the >>> mount point. I don't want to do this, since I don't want to solve this >>> in the >>> kernel at RC3 and I don't like it pretending to do things it can't do. >> >> 6. (my favorite) do NOT require that the efi/ partition be in strictly a >> fat32 format. I mean fat32 is not strictly required as the format for >> the efi >> partition. It is simply _assumed_ to be the required format and as >> such, the >> one used in so many cases. > > Wrong, see "13.3 File System Format" in UEFI specification. > it is not as simple as that:) 13.3.1.1 is more specific: = The EFI firmware must support the FAT32, FAT16, and FAT12 variants of the EFI file system. What variant of EFI FAT to use is defined by the size of the media. The rules defining the relationship between media size and FAT variants is defined in the specification for the EFI file system. We've also seen a few non-conformant systems where FAT12 support has been removed, so there's also a bit of real-world experience that goes along with reading the UEFI specification :(. I suppose that may be part of where my understanding came from. I've got a couple of "hackintoshes" that dual-boot OS X, and FreeBSD. Part of the process was working with the EFI partition (ESP) to get recognition of both OS's in the (EFI) boot menu, as well as getting the firmware properly functional in both instances. I also draw from a long article by a boot manager author whom insisted that fat32 was not a requirement. I've not got the time ATM to dig up the article. But it had many pointers to the EFI spec to prove the point. All of which I verified. In any case, sorry for the noise if I'm wrong. I'll dig up my references when I can find the time. :-) --Chris Warner On Fri, Mar 19, 2021 at 4:49 PM Toomas Soome via freebsd-current < freebsd-current@freebsd.org> wrote: ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Bug 254395] bsdinstall: fail script install after BETA3
On Fri, Mar 19, 2021 at 4:49 PM Toomas Soome via freebsd-current < freebsd-current@freebsd.org> wrote: > > > > On 20. Mar 2021, at 00:21, Yuri Pankov wrote: > > > > Chris wrote: > >> On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: > >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 > >>> > >>> Nathan Whitehorn changed: > >>> > >>>What|Removed |Added > >>> > > >>> > >>>Severity|Affects Only Me |Affects Some People > >>> CC||i...@freebsd.org > >>>Priority|--- |Normal > >>> > >>> --- Comment #6 from Nathan Whitehorn --- > >>> Thanks for the suggestion about the documentation -- I've updated the > >>> man page. > >>> > >>> The core problem here is that our tar can't extract archives to FAT32 > >>> with > >>> default options, since it treats inability to set modification time as > >>> a fatal > >>> error and FAT32 doesn't let you do that on the root directory. As > >>> such, any > >>> file in the release tarballs can't be extracted to FAT32. For > interactive > >>> installations, the bsdinstall distextract tool, a CURSES-y frontend to > >>> libarchive, solves this by ignoring ctime/mtime errors. > >>> > >>> Some extra commentary on solutions, so it can be in one place. > >>> Possibilities > >>> are: > >>> > >>> 1. We drop /boot/efi from mtree. That will result in it not existing in > >>> base.txz, solving this issue, but will result in it not being in > >>> mtree. It will > >>> also leave in place an identical bug that will break scripted > >>> installation on > >>> bare-metal POWER8 and POWER9 systems, although that is a tier-2 > platform. > >>> > >>> 2. We add an option to tar to ignore failure in setting ctime/mtime, > >>> like the > >>> interactive installer uses. This has the difficulty that the patch is > >>> hacky and > >>> would have to go through upstream. > >>> > >>> 3. We go back to using distextract for scripted installations as well > as > >>> interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. > >>> This > >>> fixes this issue but will result in installation failures for scripted > >>> installs > >>> without a controlling tty. (It will also add nice progress bars to > >>> scripted > >>> installs). > >>> > >>> 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand > >>> afterward. This is incredibly hacky and otherwise essentially > >>> functionally > >>> equivalent to #1. Like #1, it will fix this issue and has no obvious > >>> functional > >>> downside, but leaves scripted installs bare-metal POWER8 and POWER9 > >>> broken. > >>> > >>> 5. We patch the file system driver to (pretend to) allow setting times > >>> on the > >>> mount point. I don't want to do this, since I don't want to solve this > >>> in the > >>> kernel at RC3 and I don't like it pretending to do things it can't do. > >> > >> 6. (my favorite) do NOT require that the efi/ partition be in strictly > a > >> fat32 format. I mean fat32 is not strictly required as the format for > >> the efi > >> partition. It is simply _assumed_ to be the required format and as > >> such, the > >> one used in so many cases. > > > > Wrong, see "13.3 File System Format" in UEFI specification. > > > > it is not as simple as that:) > > > 13.3.1.1 is more specific: > = > The EFI firmware must support the FAT32, FAT16, and FAT12 variants of the > EFI file system. What variant of EFI FAT to use is defined by the size of > the media. The rules defining the relationship between media size and FAT > variants is defined in the specification for the EFI file system. > We've also seen a few non-conformant systems where FAT12 support has been removed, so there's also a bit of real-world experience that goes along with reading the UEFI specification :(. Warner On Fri, Mar 19, 2021 at 4:49 PM Toomas Soome via freebsd-current < freebsd-current@freebsd.org> wrote: > > > > On 20. Mar 2021, at 00:21, Yuri Pankov wrote: > > > > Chris wrote: > >> On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: > >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 > >>> > >>> Nathan Whitehorn changed: > >>> > >>>What|Removed |Added > >>> > > >>> > >>>Severity|Affects Only Me |Affects Some People > >>> CC||i...@freebsd.org > >>>Priority|--- |Normal > >>> > >>> --- Comment #6 from Nathan Whitehorn --- > >>> Thanks for the suggestion about the documentation -- I've updated the > >>> man page. > >>> > >>> The core problem here is that our tar can't extract archives to FAT32 > >>> with > >>> default options, since it treats inability to set modification time as
Re: [Bug 254395] bsdinstall: fail script install after BETA3
> On 20. Mar 2021, at 00:21, Yuri Pankov wrote: > > Chris wrote: >> On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 >>> >>> Nathan Whitehorn changed: >>> >>>What|Removed |Added >>> >>> >>>Severity|Affects Only Me |Affects Some People >>> CC||i...@freebsd.org >>>Priority|--- |Normal >>> >>> --- Comment #6 from Nathan Whitehorn --- >>> Thanks for the suggestion about the documentation -- I've updated the >>> man page. >>> >>> The core problem here is that our tar can't extract archives to FAT32 >>> with >>> default options, since it treats inability to set modification time as >>> a fatal >>> error and FAT32 doesn't let you do that on the root directory. As >>> such, any >>> file in the release tarballs can't be extracted to FAT32. For interactive >>> installations, the bsdinstall distextract tool, a CURSES-y frontend to >>> libarchive, solves this by ignoring ctime/mtime errors. >>> >>> Some extra commentary on solutions, so it can be in one place. >>> Possibilities >>> are: >>> >>> 1. We drop /boot/efi from mtree. That will result in it not existing in >>> base.txz, solving this issue, but will result in it not being in >>> mtree. It will >>> also leave in place an identical bug that will break scripted >>> installation on >>> bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. >>> >>> 2. We add an option to tar to ignore failure in setting ctime/mtime, >>> like the >>> interactive installer uses. This has the difficulty that the patch is >>> hacky and >>> would have to go through upstream. >>> >>> 3. We go back to using distextract for scripted installations as well as >>> interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. >>> This >>> fixes this issue but will result in installation failures for scripted >>> installs >>> without a controlling tty. (It will also add nice progress bars to >>> scripted >>> installs). >>> >>> 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand >>> afterward. This is incredibly hacky and otherwise essentially >>> functionally >>> equivalent to #1. Like #1, it will fix this issue and has no obvious >>> functional >>> downside, but leaves scripted installs bare-metal POWER8 and POWER9 >>> broken. >>> >>> 5. We patch the file system driver to (pretend to) allow setting times >>> on the >>> mount point. I don't want to do this, since I don't want to solve this >>> in the >>> kernel at RC3 and I don't like it pretending to do things it can't do. >> >> 6. (my favorite) do NOT require that the efi/ partition be in strictly a >> fat32 format. I mean fat32 is not strictly required as the format for >> the efi >> partition. It is simply _assumed_ to be the required format and as >> such, the >> one used in so many cases. > > Wrong, see "13.3 File System Format" in UEFI specification. > it is not as simple as that:) 13.3.1.1 is more specific: = The EFI firmware must support the FAT32, FAT16, and FAT12 variants of the EFI file system. What variant of EFI FAT to use is defined by the size of the media. The rules defining the relationship between media size and FAT variants is defined in the specification for the EFI file system. = rgds, toomas ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Bug 254395] bsdinstall: fail script install after BETA3
Chris wrote: > On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 >> >> Nathan Whitehorn changed: >> >> What |Removed |Added >> >> >> Severity|Affects Only Me |Affects Some People >> CC| |i...@freebsd.org >> Priority|--- |Normal >> >> --- Comment #6 from Nathan Whitehorn --- >> Thanks for the suggestion about the documentation -- I've updated the >> man page. >> >> The core problem here is that our tar can't extract archives to FAT32 >> with >> default options, since it treats inability to set modification time as >> a fatal >> error and FAT32 doesn't let you do that on the root directory. As >> such, any >> file in the release tarballs can't be extracted to FAT32. For interactive >> installations, the bsdinstall distextract tool, a CURSES-y frontend to >> libarchive, solves this by ignoring ctime/mtime errors. >> >> Some extra commentary on solutions, so it can be in one place. >> Possibilities >> are: >> >> 1. We drop /boot/efi from mtree. That will result in it not existing in >> base.txz, solving this issue, but will result in it not being in >> mtree. It will >> also leave in place an identical bug that will break scripted >> installation on >> bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. >> >> 2. We add an option to tar to ignore failure in setting ctime/mtime, >> like the >> interactive installer uses. This has the difficulty that the patch is >> hacky and >> would have to go through upstream. >> >> 3. We go back to using distextract for scripted installations as well as >> interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. >> This >> fixes this issue but will result in installation failures for scripted >> installs >> without a controlling tty. (It will also add nice progress bars to >> scripted >> installs). >> >> 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand >> afterward. This is incredibly hacky and otherwise essentially >> functionally >> equivalent to #1. Like #1, it will fix this issue and has no obvious >> functional >> downside, but leaves scripted installs bare-metal POWER8 and POWER9 >> broken. >> >> 5. We patch the file system driver to (pretend to) allow setting times >> on the >> mount point. I don't want to do this, since I don't want to solve this >> in the >> kernel at RC3 and I don't like it pretending to do things it can't do. > > 6. (my favorite) do NOT require that the efi/ partition be in strictly a > fat32 format. I mean fat32 is not strictly required as the format for > the efi > partition. It is simply _assumed_ to be the required format and as > such, the > one used in so many cases. Wrong, see "13.3 File System Format" in UEFI specification. > Whould it actually be that much harder to use ffs/ufs? > > You asked. ;-) > >> >> >> >> Of these, 1, 3, and 4 are quite easy to implement, but all have some >> downside. >> My temptation is to do 4 for 13.0, since it will definitely work but >> is just >> lame, then either do #2 or a variant on #3 where distextract notices >> there is >> no tty and doesn't try to set up a dialog as a longer-term fix in >> HEAD. Any >> thoughts? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Bug 254395] bsdinstall: fail script install after BETA3
On 2021-03-19 13:06, bugzilla-nore...@freebsd.org wrote: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Nathan Whitehorn changed: What|Removed |Added Severity|Affects Only Me |Affects Some People CC||i...@freebsd.org Priority|--- |Normal --- Comment #6 from Nathan Whitehorn --- Thanks for the suggestion about the documentation -- I've updated the man page. The core problem here is that our tar can't extract archives to FAT32 with default options, since it treats inability to set modification time as a fatal error and FAT32 doesn't let you do that on the root directory. As such, any file in the release tarballs can't be extracted to FAT32. For interactive installations, the bsdinstall distextract tool, a CURSES-y frontend to libarchive, solves this by ignoring ctime/mtime errors. Some extra commentary on solutions, so it can be in one place. Possibilities are: 1. We drop /boot/efi from mtree. That will result in it not existing in base.txz, solving this issue, but will result in it not being in mtree. It will also leave in place an identical bug that will break scripted installation on bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. 2. We add an option to tar to ignore failure in setting ctime/mtime, like the interactive installer uses. This has the difficulty that the patch is hacky and would have to go through upstream. 3. We go back to using distextract for scripted installations as well as interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. This fixes this issue but will result in installation failures for scripted installs without a controlling tty. (It will also add nice progress bars to scripted installs). 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand afterward. This is incredibly hacky and otherwise essentially functionally equivalent to #1. Like #1, it will fix this issue and has no obvious functional downside, but leaves scripted installs bare-metal POWER8 and POWER9 broken. 5. We patch the file system driver to (pretend to) allow setting times on the mount point. I don't want to do this, since I don't want to solve this in the kernel at RC3 and I don't like it pretending to do things it can't do. 6. (my favorite) do NOT require that the efi/ partition be in strictly a fat32 format. I mean fat32 is not strictly required as the format for the efi partition. It is simply _assumed_ to be the required format and as such, the one used in so many cases. Whould it actually be that much harder to use ffs/ufs? You asked. ;-) Of these, 1, 3, and 4 are quite easy to implement, but all have some downside. My temptation is to do 4 for 13.0, since it will definitely work but is just lame, then either do #2 or a variant on #3 where distextract notices there is no tty and doesn't try to set up a dialog as a longer-term fix in HEAD. Any thoughts? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #11 from Nathan Whitehorn --- (In reply to Warner Losh from comment #10) It's pretty tricky, since it touches code in a lot of places, has to be conditional on platform, and runs a risk of breaking interactive installs just because it is pretty invasive. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #10 from Warner Losh --- (In reply to Ryan Moeller from comment #8) Oh! I like this idea better, I think, but I don't know how hard it is to do. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #9 from Nathan Whitehorn --- (In reply to Ryan Moeller from comment #8) We could delay it, but it's harder and less-localized than the other solutions. It also completely breaks PowerPC and other systems with analagous but non-ESP boot partions. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #8 from Ryan Moeller --- Would it be possible to defer mounting the ESP until after the tarballs have been extracted? We don't actually have anything in /boot/efi in the base archive. My understanding is that it's just there to ensure the mountpoint exists. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #7 from Warner Losh --- #4 sounds good for 13.0. Of the hacks, it's the most localized and least hacky, imho. I'd prefer #5, honestly, longer term but I've not thought through all the implictions. We should loop in cem@ since he's been touching that code most recently. Or delphij@ as he has too... -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Nathan Whitehorn changed: What|Removed |Added Severity|Affects Only Me |Affects Some People CC||i...@freebsd.org Priority|--- |Normal --- Comment #6 from Nathan Whitehorn --- Thanks for the suggestion about the documentation -- I've updated the man page. The core problem here is that our tar can't extract archives to FAT32 with default options, since it treats inability to set modification time as a fatal error and FAT32 doesn't let you do that on the root directory. As such, any file in the release tarballs can't be extracted to FAT32. For interactive installations, the bsdinstall distextract tool, a CURSES-y frontend to libarchive, solves this by ignoring ctime/mtime errors. Some extra commentary on solutions, so it can be in one place. Possibilities are: 1. We drop /boot/efi from mtree. That will result in it not existing in base.txz, solving this issue, but will result in it not being in mtree. It will also leave in place an identical bug that will break scripted installation on bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. 2. We add an option to tar to ignore failure in setting ctime/mtime, like the interactive installer uses. This has the difficulty that the patch is hacky and would have to go through upstream. 3. We go back to using distextract for scripted installations as well as interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. This fixes this issue but will result in installation failures for scripted installs without a controlling tty. (It will also add nice progress bars to scripted installs). 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand afterward. This is incredibly hacky and otherwise essentially functionally equivalent to #1. Like #1, it will fix this issue and has no obvious functional downside, but leaves scripted installs bare-metal POWER8 and POWER9 broken. 5. We patch the file system driver to (pretend to) allow setting times on the mount point. I don't want to do this, since I don't want to solve this in the kernel at RC3 and I don't like it pretending to do things it can't do. Of these, 1, 3, and 4 are quite easy to implement, but all have some downside. My temptation is to do 4 for 13.0, since it will definitely work but is just lame, then either do #2 or a variant on #3 where distextract notices there is no tty and doesn't try to set up a dialog as a longer-term fix in HEAD. Any thoughts? -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Andrey Fesenko changed: What|Removed |Added Blocks||231027 Referenced Bugs: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231027 [Bug 231027] [META] FreeBSD-Foundation sponsored issues for FreeBSD 13-CURRENT -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #5 from Andrey Fesenko --- (In reply to Nathan Whitehorn from comment #3) If possible, it is also worth mentioning in the release notes that you no longer need to specify the efi and boot partitions, but is it possible to make the efi partition smaller, since 260M is a lot for a virtual machine, where there will be only 1 system -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395
--- Comment #4 from Andrey Fesenko ---
(In reply to Nathan Whitehorn from comment #1)
Installer fail, if set PARTITIONS="$DISKSLICE GPT { auto freebsd-ufs / }" fail
too
https://64.media.tumblr.com/2c95d2ddeed5cc2bd7ffb8fc68f34436/f8abb8c6d81ea645-e4/s1280x1920/6c4753c8bfdd3af75f597371189c4dcb1cbf0880.png
build packer
"qemu_binary": "/usr/libexec/qemu-kvm",
"-bios",
"/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd"
Check virtualbox builder BETA4 installer ok. RC2 fail, similar error
--
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 --- Comment #3 from Nathan Whitehorn --- Well, I found the issue and there's even a nice comment in the relevant code that an older, smarter version of me put in in 2018 describing exactly why this is going to break: # Unpack distributions bsdinstall checksum for set in $DISTRIBUTIONS; do f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set" # XXX: this will fail if any mountpoints are FAT, due to inability to # set ctime/mtime on the root of FAT partitions. tar has no option to # ignore this. We probably need to switch back to distextract here # to properly support EFI. tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT done I'll try to get a patch in today. Apologies for the breakage, and thanks for the report. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Ryan Moeller changed: What|Removed |Added CC||freql...@freebsd.org --- Comment #2 from Ryan Moeller --- This affects me as well. The install completely fails. I'm even doing EFI installs: PARTITIONS="zvol/pool/dataset/vol gpt" I've used this same script quite a lot and it has only started to fail recently. I noticed yesterday and started looking into it a bit before I got pulled into something else. The tarball that fails to be extracted is base.txz (it's the one that contains ./boot/efi). The output on the console shows this as well: DEBUG: dialog.subr: DEBUG_SELF_INITIALIZE=[] DEBUG: UNAME_S=[FreeBSD] UNAME_P=[amd64] UNAME_R=[14.0-CURRENT] DEBUG: common.subr: Successfully loaded. DEBUG: f_debug_init: ARGV=[mount] GETOPTS_STDARGS=[dD:] DEBUG: f_debug_init: debug=[1] debugFile=[] DEBUG: Running installation step: mount DEBUG: dialog.subr: DEBUG_SELF_INITIALIZE=[] DEBUG: UNAME_S=[FreeBSD] UNAME_P=[amd64] UNAME_R=[14.0-CURRENT] DEBUG: common.subr: Successfully loaded. DEBUG: f_debug_init: ARGV=[checksum] GETOPTS_STDARGS=[dD:] DEBUG: f_debug_init: debug=[1] debugFile=[] DEBUG: Running installation step: checksum DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/kernel.txz DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/kernel-dbg.txz DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/base.txz ERROR: bsdinstall script failed At this point the script exits, leaving the filesystems mounted even: /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 on /mnt (ufs, local, journaled soft-updates) /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 on /mnt/boot/efi (msdosfs, local) devfs on /mnt/dev (devfs) # cat /tmp/bsdinstall-tmp-fstab /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 /mnt/boot/efi msdosfs rw 2 2 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 /mntufs rw 1 1 # cat /tmp/bsdinstall_etc/fstab # DeviceMountpoint FStype Options DumpPass# /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 /boot/efi msdosfs rw 2 2 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 / ufs rw 1 1 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep3 none swapsw 0 0 -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395 Nathan Whitehorn changed: What|Removed |Added CC||nwhiteh...@freebsd.org --- Comment #1 from Nathan Whitehorn --- You should not include the EFI partition in the PARTITIONS variable in general, but that won't cause this issue. Does the installer actually fail, resulting in a non-bootable system? Or do you just see this message in the output? Any more detail about your install script? What are you untarring? -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[Bug 254395] bsdinstall: fail script install after BETA3
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254395
Bug ID: 254395
Summary: bsdinstall: fail script install after BETA3
Product: Base System
Version: 13.0-STABLE
Hardware: Any
OS: Any
Status: New
Keywords: regression
Severity: Affects Only Me
Priority: ---
Component: bin
Assignee: b...@freebsd.org
Reporter: and...@bsdnir.info
CC: curr...@freebsd.org
After BETA3 or RC1 (CURRENT add efi partition /boot/efi mount) script install
UEFI system fail checksum/extract base.txz
Old working `PARTITIONS="$DISKSLICE GPT { 260M efi , auto freebsd-ufs / }"`
if edit `PARTITIONS="$DISKSLICE GPT { 260M efi /boot/efi, auto freebsd-ufs /
}"` not fix
root /mnt and efi /mnt/boot/efi mount succesfull
bsdinstall_log
* DEBUG: f_variable_set_defaults: Initializing defaults...
* DEBUG: f_getvar: var=[debug] value=[1] r=0
* DEBUG: f_getvar: var=[editor] value=[/usr/bin/ee] r=0
* DEBUG: f_getvar: var=[ftpState] value=[auto] r=0
* DEBUG: f_getvar: var=[ftpUser] value=[ftp] r=0
* DEBUG: f_getvar: var=[hostname] value=[] r=0
* DEBUG: f_getvar: var=[MEDIA_TIMEOUT] value=[300] r=0
* DEBUG: f_getvar: var=[nfs_reserved_port_only] value=[NO] r=0
* DEBUG: f_getvar: var=[nfs_use_tcp] value=[NO] r=0
* DEBUG: f_getvar: var=[nfs_use_v3] value=[YES] r=0
* DEBUG: f_getvar: var=[PKG_TMPDIR] value=[/var/tmp] r=0
* DEBUG: f_getvar: var=[releaseName] value=[13.0-RC2] r=0
* DEBUG: f_variable_set_defaults: Defaults initialized.
* DEBUG: variable.subr: Successfully loaded.
* DEBUG: f_include_lang: file=[/usr/libexec/bsdconfig/include/messages.subr]
lang=[]
* DEBUG: dialog.subr: DIALOG_SELF_INITIALIZE=[1]
* DEBUG: f_dialog_init: ARGV=[/tmp/installerconfig] GETOPTS_STDARGS=[dD:SX]
* DEBUG: f_dialog_init: SECURE=[] USE_XDIALOG=[]
* DEBUG: f_dialog_init: dialog(1) API initialized.
* DEBUG: dialog.subr: Successfully loaded.
* DEBUG: f_include: file=[/usr/share/bsdconfig/variable.subr]
* DEBUG: Began Installation at Fri Mar 19 02:37:31 UTC 2021
* ./boot/efi/: Can't restore time
* tar: Error exit delayed from previous errors.
--
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
