Re: How to create a custom Debian ISO

[Aditya Garg]

> On 19 May 2024, at 5:57 PM, Thomas Schmitt  wrote:
> 
> Hi,
> 
> in the second part of this mail i discuss a possible problem with script
> 04_create_iso.sh about bootability on Legacy BIOS from USB stick.
> (I apologize already now for being off topic by talking about an Ubuntu
> ISO.)
> 
> 
> Roland Clobus wrote:
>> Thanks for pointing to live-build :-)
> 
> Just connecting my users. :))
> 
> 
 [1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
> I wrote:
>>> Just out of pure curiosity: From what script in particular do you get
>>> this impression ?
> 
> Roland Clobus wrote:
>> The scripts with the sequence numbers 01-04 mirror the steps that are done
>> in live-build,
> 
> 01_build_file_system.sh looks quite debianish. (Not my turf.)
> 02_build_image.sh mentions squashfs, which is indeed typical for Live ISOs.
> But as said, with Ubuntu desktop there is no non-Live ISO.
> 03_prepare_iso.sh prepares bootloader files.
> (I know the topics of 02 and 03 only as bystander.)
> 
> --

Well, most of the code in that repo was written by the previous Ubuntu 
maintainer.
I don’t know what he thought at that time, but yes, now you have mentioned,
this looks kinda broken to me.

But T2 Macs cannot boot via MBR code, then only support UEFI. So even I never
bothered messing up with this code, as it never caused any issues.


> Now for the announced discussion of a possible problem in the production
> of derived Ubuntu ISOs:
> 
> 04_create_iso.sh looks on the first glimpse like producing a usual amd64
> hybrid ISO for {Legacy,EFI} x {DVD,HDD}, but in detail it is somewhat
> riddling:
> 
>  20   --grub2-mbr "/usr/lib/grub/i386-pc/boot_hybrid.img" \
>  ...
>  24   -isohybrid-mbr "${ROOT_PATH}/files/isohdpfx.bin" \
> 
> First xorriso is instructed to use boot_hybrid.img as MBR with some extra
> info for GRUB.
> But then this is overridden by the instruction to use isohdpfx.bin as MBR
> with some extra info for ISOLINUX/SYSLINUX.
> 
> In my local experiments this leads to dysfunctional MBR code because
> -isohybrid-mbr works its magic only if a ISOLINUX boot image for El Torito
> is present. But i understand from 03_prepare_iso.sh that the image file
> given by option -b is despite its name "isolinux/bios.img" concatenated
> from GRUB files cdboot.img and core.img.
> 
> I wonder whether "T2 Macs" would be able to boot via MBR code at all.
> That would be EFI in CSM mode and ISO on USB stick.
> Does this work with the currently produced ISOs ?
> 
> Obviously i am not smart enough to find one of the resulting ISOs in the
> web for inspection. Shrug.
> 
> Can it be that -isohybrid-mbr was used because else
>  25   -isohybrid-gpt-basdat -isohybrid-apm-hfsplus \
> would cause an error ?
> If so, then option
> 
>  -part_like_isohybrid
> 
> might be the better alternative because it does not override the
> preparation for using boot_hybrid.img as --grub2-mbr.
> If i replace in my experiment the line
>  -isohybrid-mbr "${ROOT_PATH}/files/isohdpfx.bin" \
> by -part_like_isohybrid, then i get an ISO where xorriso can recognize
> an MBR with the patched-in extra info caused by option --grub2-mbr.
> The partition table is more sparse than with a real ISOLINUX MBR. But
> it marks the EFI partition in MBR and (pseudo-)GPT.
> 
> 
> Whatever, current Ubuntu desktop ISOs do not use -isohybrid-gpt-basdat
> and -isohybrid-apm-hfsplus any more.
> One may run e.g.:
> 
>  xorriso -indev ubuntu-22.04.3-desktop-amd64.iso -report_el_torito as_mkisofs
> 
> to see -append_partition , -appended_part_as_gpt ,
> and -e '--interval:appended_partition_2:all::' serving instead of
> -part_like_isohybrid (or -isohybrid-mbr), -e "EFI/efiboot.img", and
> -isohybrid-gpt-basdat .
> 
> 
> Have a nice day :)
> 
> Thomas
> 

Re: How to create a custom Debian ISO

[Roland Clobus]

Hello Aditya,

On 19/05/2024 13:45, Aditya Garg wrote:
I also happed to come across this guide 
https://www.willhaley.com/blog/custom-debian-live-environment/ 
.


Is it worth trying? It's quite similar to Ubuntu ISOs.


Well, I'm biased here, being one of the maintainers of live-build... So 
I would suggest not to follow those steps, because you would 
re-implement a lot that is already performed by live-build, with the 
risk of falling into all the traps that have already been solved.


Those instructions strip down all the steps that live-build does down to 
the barest minimum.
When using live-build, I suggest you use the git version (not the 
packaged version) as detailed on [1].


You will get:
* An installer in the boot menu (if you require that)
* Have a reproducible image (bit-for-bit identical if re-built at a 
later moment)
* The benefit of using a package that is used by several distros instead 
of being the only distro using your own scripts, which means that there 
will be less bugs


With kind regards,
Roland Clobus

[1] https://wiki.debian.org/ReproducibleInstalls/LiveImages


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: How to create a custom Debian ISO

[Thomas Schmitt]

Hi,

in the second part of this mail i discuss a possible problem with script
04_create_iso.sh about bootability on Legacy BIOS from USB stick.
(I apologize already now for being off topic by talking about an Ubuntu
ISO.)


Roland Clobus wrote:
> Thanks for pointing to live-build :-)

Just connecting my users. :))


> > > [1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
I wrote:
> > Just out of pure curiosity: From what script in particular do you get
> > this impression ?

Roland Clobus wrote:
> The scripts with the sequence numbers 01-04 mirror the steps that are done
> in live-build,

01_build_file_system.sh looks quite debianish. (Not my turf.)
02_build_image.sh mentions squashfs, which is indeed typical for Live ISOs.
But as said, with Ubuntu desktop there is no non-Live ISO.
03_prepare_iso.sh prepares bootloader files.
(I know the topics of 02 and 03 only as bystander.)

--
Now for the announced discussion of a possible problem in the production
of derived Ubuntu ISOs:

04_create_iso.sh looks on the first glimpse like producing a usual amd64
hybrid ISO for {Legacy,EFI} x {DVD,HDD}, but in detail it is somewhat
riddling:

  20   --grub2-mbr "/usr/lib/grub/i386-pc/boot_hybrid.img" \
  ...
  24   -isohybrid-mbr "${ROOT_PATH}/files/isohdpfx.bin" \

First xorriso is instructed to use boot_hybrid.img as MBR with some extra
info for GRUB.
But then this is overridden by the instruction to use isohdpfx.bin as MBR
with some extra info for ISOLINUX/SYSLINUX.

In my local experiments this leads to dysfunctional MBR code because
-isohybrid-mbr works its magic only if a ISOLINUX boot image for El Torito
is present. But i understand from 03_prepare_iso.sh that the image file
given by option -b is despite its name "isolinux/bios.img" concatenated
from GRUB files cdboot.img and core.img.

I wonder whether "T2 Macs" would be able to boot via MBR code at all.
That would be EFI in CSM mode and ISO on USB stick.
Does this work with the currently produced ISOs ?

Obviously i am not smart enough to find one of the resulting ISOs in the
web for inspection. Shrug.

Can it be that -isohybrid-mbr was used because else
  25   -isohybrid-gpt-basdat -isohybrid-apm-hfsplus \
would cause an error ?
If so, then option

  -part_like_isohybrid

might be the better alternative because it does not override the
preparation for using boot_hybrid.img as --grub2-mbr.
If i replace in my experiment the line
  -isohybrid-mbr "${ROOT_PATH}/files/isohdpfx.bin" \
by -part_like_isohybrid, then i get an ISO where xorriso can recognize
an MBR with the patched-in extra info caused by option --grub2-mbr.
The partition table is more sparse than with a real ISOLINUX MBR. But
it marks the EFI partition in MBR and (pseudo-)GPT.


Whatever, current Ubuntu desktop ISOs do not use -isohybrid-gpt-basdat
and -isohybrid-apm-hfsplus any more.
One may run e.g.:

  xorriso -indev ubuntu-22.04.3-desktop-amd64.iso -report_el_torito as_mkisofs

to see -append_partition , -appended_part_as_gpt ,
and -e '--interval:appended_partition_2:all::' serving instead of
-part_like_isohybrid (or -isohybrid-mbr), -e "EFI/efiboot.img", and
-isohybrid-gpt-basdat .


Have a nice day :)

Thomas

Re: How to create a custom Debian ISO

[Aditya Garg]

That's great Roland!

I also happed to come across this guide 
https://www.willhaley.com/blog/custom-debian-live-environment/.

Is it worth trying? It's quite similar to Ubuntu ISOs.

On 19 May 2024, at 5:12 PM, Roland Clobus  wrote:

Hello Aditya,

On 19/05/2024 13:08, Aditya Garg wrote:
Just to get a clarification, would the user be able to install Debian in a live 
build as well?

Yes, the current Debian live images contain both a Debian-Installer and 
Calamares which will install the live images to your hard disk.
Additionally (not well tested yet) it can run the Debian-Installer similar to 
the Debian-Installer from the netinst image, i.e. it will perform a full 
customised installation.

You can find weekly builds at [1], with the test results at [2].

With kind regards,
Roland Clobus

[1] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/
[2] https://openqa.debian.net/group_overview/14?only_tagged=1

PS: No need to CC me, I'm subscribed to the lists

Re: How to create a custom Debian ISO

[Roland Clobus]

Hello Aditya,

On 19/05/2024 13:08, Aditya Garg wrote:
Just to get a clarification, would the user be able to install Debian in 
a live build as well?


Yes, the current Debian live images contain both a Debian-Installer and 
Calamares which will install the live images to your hard disk.
Additionally (not well tested yet) it can run the Debian-Installer 
similar to the Debian-Installer from the netinst image, i.e. it will 
perform a full customised installation.


You can find weekly builds at [1], with the test results at [2].

With kind regards,
Roland Clobus

[1] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/
[2] https://openqa.debian.net/group_overview/14?only_tagged=1

PS: No need to CC me, I'm subscribed to the lists


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: How to create a custom Debian ISO

[Aditya Garg]

Hi Roland

Just to get a clarification, would the user be able to install Debian in a live 
build as well?

And just in case Marvin is in the thread, I'd like to say sorry for the 
double-post, as mentioned in 
https://lists.debian.org/debian-user/2024/05/msg00149.html. Actually the 
double-post was due to a bug in my email client.

On 19 May 2024, at 3:42 PM, Roland Clobus  wrote:

Hello Thomas,

On 18/05/2024 14:23, Thomas Schmitt wrote:
since Aditya Garg gets a reply here at debian-live, i add a link to the
thread on debian-user, beginning with Marvin Renich's proposal to
continue the thread there:
  https://lists.debian.org/debian-user/2024/05/msg00149.html
(I proposed for Debian Live
  https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
and for Debian installation ISOs
  https://wiki.debian.org/RepackBootableISO
)

Thanks for pointing to live-build :-) I saw the discussion originally on 
debian-devel, and it switched several times (Also I switched it away from 
debian-devel)

Roland Clobus wrote:
I had a quick peek at the scripts you use [1].
[1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
It appears to me that you want to create a custom Debian Live ISO (not a
netinst image).
Just out of pure curiosity: From what script in particular do you get
this impression ?

The scripts with the sequence numbers 01-04 mirror the steps that are done in 
live-build, with the difference that live-build is more complex/configurable, 
due to it multi-purpose-tool character.

With kind regards,
Roland Clobus

Re: How to create a custom Debian ISO

[Roland Clobus]

Hello Thomas,

On 18/05/2024 14:23, Thomas Schmitt wrote:

since Aditya Garg gets a reply here at debian-live, i add a link to the
thread on debian-user, beginning with Marvin Renich's proposal to
continue the thread there:

   https://lists.debian.org/debian-user/2024/05/msg00149.html

(I proposed for Debian Live
   https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
and for Debian installation ISOs
   https://wiki.debian.org/RepackBootableISO
)


Thanks for pointing to live-build :-) I saw the discussion originally on 
debian-devel, and it switched several times (Also I switched it away 
from debian-devel)



Roland Clobus wrote:

I had a quick peek at the scripts you use [1].
[1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
It appears to me that you want to create a custom Debian Live ISO (not a
netinst image).


Just out of pure curiosity: From what script in particular do you get
this impression ?


The scripts with the sequence numbers 01-04 mirror the steps that are 
done in live-build, with the difference that live-build is more 
complex/configurable, due to it multi-purpose-tool character.


With kind regards,
Roland Clobus


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: How to create a custom Debian ISO

[Thomas Schmitt]

Hi,

since Aditya Garg gets a reply here at debian-live, i add a link to the
thread on debian-user, beginning with Marvin Renich's proposal to
continue the thread there:

  https://lists.debian.org/debian-user/2024/05/msg00149.html

(I proposed for Debian Live
  https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
and for Debian installation ISOs
  https://wiki.debian.org/RepackBootableISO
)


Roland Clobus wrote:
> I had a quick peek at the scripts you use [1].
> [1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
> It appears to me that you want to create a custom Debian Live ISO (not a
> netinst image).

Just out of pure curiosity: From what script in particular do you get
this impression ?

AFAIK, Ubuntu desktop ISOs are always Live systems.
  https://releases.ubuntu.com/noble/
says
  "The desktop image allows you to try Ubuntu without changing your
   computer at all, and at your option to install it permanently later."


Have a nice day :)

Thomas

Re: How to create a custom Debian ISO

[Roland Clobus]

+debian-live

Hello Aditya,

On 11/05/2024 10:21, Aditya Garg wrote:

I wanted to create a custom ISO of Debian, with the following customisations:

1. I want to add a custom kernel that supports my Hardware.
2. I want to add my own Apt repo which hosts various software packages to 
support my hardware.

I am not able to get any good documentation for the same. Please help.


Let me chime in on this thread as well.
I had a quick peek at the scripts you use [1].

It appears to me that you want to create a custom Debian Live ISO (not a 
netinst image). If so, you can take a look at live-build, which is used 
for the Debian live images.
Steps describing how to build a live ISO image are at [2] and a generic 
manual for live-build is at [3].


You can then build an ISO image from their packages, without the need 
for repacking (and automagically all checksums will match)


With kind regards,
Roland Clobus
Co-maintainer of live-build

[1] https://github.com/t2linux/T2-Ubuntu/tree/LTS
[2] https://wiki.debian.org/ReproducibleInstalls/LiveImages
[3] 
https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html




OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: How to create a custom Debian ISO

[Aditya Garg]

Well it's indeed not as easy as I thought as far as Debian ISOs are concerned.

I'll try to be more precise. I am a maintainer for Ubuntu on Linux on T2 Macs 
project: https://t2linux.org/.

We work to modify ISOs of commonly used distros by adding a custom kernel with 
drivers for T2 Macs and provide to the users. There has been a demand for 
Debian for a long time and I wish to provide the ISOs for the same. I would 
prefer making the ISO as similar to the official Debian ISO and just replace 
the Debian kernel with the customised kernel.

I've already set up an apt repo which hosts the kernels over here: 
https://github.com/AdityaGarg8/t2-ubuntu-repo

I'll be thankful if I get the best possible option.

On 11 May 2024, at 8:33 PM, Thomas Schmitt  wrote:

Hi,

Aditya Garg wrote to debian-devel:
I wanted to create a custom ISO of Debian, with the following customisations:
1. I want to add a custom kernel that supports my Hardware.
2. I want to add my own Apt repo which hosts various software packages to
support my hardware.
I am not able to get any good documentation for the same. Please help.

Marvin Renich wrote:
The package live-build from the Debian Live project might help you do
what you want.

Indeed the live-build package seems to be in use outside Debian's own
ISO production. Mailing list is
 debian-live@lists.debian.org
There exists a manual
 https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html

Installation ISOs are made by package debian-cd, of which i am not aware
that it would have have users outside the official ISO production.i
Mailing list is
 debian...@lists.debian.org
Your impression about lack of documentation is not wrong as far as this
project is concerned. :))


Nevertheless the production step of packing up the ISO from a prepared
file tree is documented together with methods to use a Debian installation
ISO as base for the preparation:
 https://wiki.debian.org/RepackBootableISO

Packages may probably be added at the appropriate place in the directory
tree under /pool. (Managing a Debian repo is not my turf. Sorry for
being vague here.)

Changing the content of a Debian ISO might need some follow-up work in
administrative files of the ISO.
When merging Debian ISOs, my script
 
https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/test/merge_debian_isos
manipulates:
 /README.txt
 /dists/*/Release
and merges the files listed in /dists/*/Release.
You would have to explore whether these files are affected by your
changes.


Have a nice day :)

Thomas