Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-16 Thread Thomas Schmitt
Hi, Pete Batard wrote: > Debian does not use an efi.img. Oh it does with ISOs for i386 and amd64. There is a data file in the ISO filesystem named /boot/grub/efi.img advertised as MBR partition of type 0xEF and as El Torito boot image for EFI: $ xorriso -indev debian-11.5.0-amd64-netinst.iso

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-16 Thread Pete Batard
Hi, Just going to report that I have tested media creation from the 2023-03-13 version of debian-testing-amd64-netinst.iso, using only the Windows native utilities (i.e. formatting a USB drive to FAT32 using Windows Disk manager and mounting the ISO and copying the files using Windows File Ex

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-15 Thread Thomas Schmitt
Hi, James Addison wrote: > The problem, in both cases, was that I hadn't copied the '.disk' dotfile > directory from the install media ISO filesystem(s) in each case. Besides such user pitfalls with the produced ISO and the problem of symbolic links there are other constraints which an ISO has to

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-15 Thread Pete Batard
Hi James On 2023.03.15 00:47, James Addison wrote: The problem, in both cases, was that I hadn't copied the '.disk' dotfile directory from the install media ISO filesystem(s) in each case. Ah, yes, the infamous '.disk/' directory. If it's any consolation, you're not the first person to stumbl

Re: Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-15 Thread Pete Batard
Please disregard my previous message to this list, that was a bug specific reply sent to the wrong e-mail. /Pete

Re: Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-14 Thread Pete Batard
Hi James, On 2023.03.15 00:47, James Addison wrote: The problem, in both cases, was that I hadn't copied the '.disk' dotfile directory from the install media ISO filesystem(s) in each case. Ah, yes, the infamous '.disk/' directory. If it's any consolation, you're not the first person to stumb

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-14 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 On Wed, 15 Mar 2023 00:04:28 +, James Addison wrote: > I haven't (have not) been able to install Debian from FAT32-based media > created > using file-copy from either of these two approaches. > In both cases d-i integrity checks of the FAT32 medi

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-14 Thread James Addison
Followup-For: Bug #1031696 On Mon, 13 Mar 2023 01:04:26 +, James Addison wrote: > I've been able to build local NETINST ISO images successfully using the merged > changes (thanks, Steve), and was also able to file-copy them to a FAT32 > filesystem and successfully boot from that into Debian In

Re: Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-14 Thread Pete Batard
On 2023.03.13 01:04, James Addison wrote: Pete: perhaps it'd be worth filing a separate bug for the documentation-copy issue? It'd be good to understand what the practical implications of that are. I have now created https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1032941. Personally, I d

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread James Addison
Followup-For: Bug #1031696 Control: tags -1 - moreinfo I've been able to build local NETINST ISO images successfully using the merged changes (thanks, Steve), and was also able to file-copy them to a FAT32 filesystem and successfully boot from that into Debian Installer using QEMU. Building imag

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 > >Thinking aloud: as an alternative, would adding the '-f' flag to MKISOFS > >achieve the desired result for both documentation and firmware files, without > >requiring any other changes? > No, then I expect we'll simply end up with duplicate copies

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread Thomas Schmitt
Hi, James Addison wrote: > Thinking aloud: as an alternative, would adding the '-f' flag to MKISOFS > achieve the desired result for both documentation and firmware files, > without requiring any other changes? Let's pack up some symbolic links with -f: $ sudo mount firmware-bookworm-DI-alpha1

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread Steve McIntyre
On Sun, Mar 12, 2023 at 05:43:24PM +, James Addison wrote: >Followup-For: Bug #1031696 > >Thinking aloud: as an alternative, would adding the '-f' flag to MKISOFS >achieve the desired result for both documentation and firmware files, without >requiring any other changes? No, then I expect we'l

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread James Addison
Followup-For: Bug #1031696 Thinking aloud: as an alternative, would adding the '-f' flag to MKISOFS achieve the desired result for both documentation and firmware files, without requiring any other changes? (I'll mention as context that there are symlinks in the debian-faq tarball that is used as

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread Thomas Schmitt
Hi, James Addison wrote: > Please find below a proposed update to the commit message under discussion: Only one nitpick: > multiple filenames ("records"). The ISO9660 generation tools > genisoimage and xorriso can use this to de-duplicate multiple linked > files from their input, sav

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 X-Debbugs-Cc: scdbac...@gmx.net Please find below a proposed update to the commit message under discussion: nonfree-firmware: when creating on-disc firmware links, use the same link creation logic as archive-area links This changeset swi

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-12 Thread Thomas Schmitt
Hi, James Addison wrote: > I've opened a merge request at > https://salsa.debian.org/images-team/debian-cd/-/merge_requests/30 > [...] > The commit message contains an effort to explain what's going on; please > consider that message as reviewable and open to feedback too. I'm clumsy with web bas

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 Thank you again Thomas. I've opened a merge request at https://salsa.debian.org/images-team/debian-cd/-/merge_requests/30 My Debian mirror is still under construction, hence the 'draft' status for the merge request (to indicate that I'm not certain

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread Thomas Schmitt
Hi, James Addison wrote: > My interpretation of the commands and output in your comment is that both > genisoimage and xorriso can translate hardlinks from a source filesystem > into deduplicated file references in a written ISO filesystem With genisoimage we only know empirically. With libisofs

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Followup-For: Bug #1031696 On Sat, 11 Mar 2023 18:10:22 +0100, Thomas Schmitt wrote: > ifneq (,$(filter i386 amd64 arm64 hppa,$(ARCHES))) > Surely i386, amd64, and arm64 get their published Debian ISOs made > by xorriso. I think your expectation is correct there, yes: looking further into the

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread Thomas Schmitt
Hi, James Addison wrote: > it looks like the selection of CD image > creation tool is configured per-architecture here: > https://salsa.debian.org/images-team/debian-cd/-/blob/5aebb6794a3b8b2393663fb643e35eb8e510c9a4/Makefile#L24 I wish i would understand the clause ifneq (,$(filter i386 amd64

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Followup-For: Bug #1031696 On Sat, 11 Mar 2023 12:16:47 +0100, Thomas Schmitt wrote: > This could be achieved by a bunch of hard links instead of the symbolic > links. In /firmware of firmware-bookworm-DI-alpha1-amd64-netinst.iso > i see only symbolic links to data files. So replacing symlinks by

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread Pete Batard
Hi James, > so the regression: > > * IS NOT that non-free firmware becomes unavailable when UEFI > file transposition is used to create bootable drives > -- because that firmware was not available previously > > * IS that bootable drives created using UEFI file transposition > fro

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread Thomas Schmitt
Hi, James Addison wrote: > Something that symlinks can do is to allow image creators to save space by > by de-duplicating files. [...] This could be achieved by a bunch of hard links instead of the symbolic links. In /firmware of firmware-bookworm-DI-alpha1-amd64-netinst.iso i see only symbolic l

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 Hi Pete - could you check my understanding of your paragraph: > In short, if this issue is left unaddressed, bookworm will be > introducing a *regression* compared to bullseye, in that it will no > longer be possible to perform a Debian installation

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Package: debian-cd Followup-For: Bug #1031696 On Sat, 11 Mar 2023 03:31:50 +, Pete wrote: > Just going to add, in case you wonder why it shouldn't be up to the > software that is mounting the ISO to sort out symbolic links and just > duplicate content, that neither Windows File Explorer nor

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-11 Thread James Addison
Followup-For: Bug #1031696 X-Debbugs-Cc: p...@akeo.ie Control: reassign -1 debian-cd Control: tags -1 moreinfo I forgot to mention: thanks, Pete, for testing this UEFI file transposition support with bookworm's d-i alpha 2. We're all volunteers and I'm no expert with the CD image build process,

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-10 Thread Pete Batard
Thanks for looking into this. Just going to add, in case you wonder why it shouldn't be up to the software that is mounting the ISO to sort out symbolic links and just duplicate content, that neither Windows File Explorer nor 7-zip (both of which can mount/extract ISO content) will list anythi

Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

2023-03-10 Thread James Addison
Package: cdimage.debian.org Followup-For: Bug #1031696 X-Debbugs-Cc: p...@akeo.ie My best guess at the moment is that the relevant section of the CD image preparation scripts is: https://salsa.debian.org/images-team/debian-cd/-/blob/5aebb6794a3b8b2393663fb643e35eb8e510c9a4/tools/make_disc_trees.p