Re: [OE-core] [PATCH 0/4] add wic based image installer
Thanks! FWIW, I really think an installer improvement would be go to
from "copy image contents to install media, update slightly" to
"write compressed image to disk (use bmaptool to handle decomp), update
a few things like UUIDs". This would buy us a lot of space as a .xz
compressed image is going to tend to be a lot smaller than 4GB, even
with a ton of things in it.
On Tue, May 21, 2019 at 07:46:52AM +, Lee, Chee Yang wrote:
> Tom, Thanks for the sharing
> I try out this method and it works
> however I notice that sdimage-bootpart.wks format bootimg-partition with vfat
> still limit the rootfs.img size to 4GB.
> Anyway, with the similar trick and using a new .wks with combination of 2
> partition below could build the installer image for image greater than 4GB.
> - bootimg-efi with "wic: bootimg-efi: add label source parameter" patch
> - bootimg-partition ( format in ext4 )
> so I think it would be great to keep the enhancement for bootimg-efi while
> ignore the other 3 patches.
>
>
> On 5/16/19, 8:20 PM, "Tom Rini" wrote:
>
> On Thu, May 16, 2019 at 11:48:02AM +, Lee, Chee Yang wrote:
>
> > I couldn’t figure out how to make it with sdimage-bootpart.wks, can
> > you tell me what did you set for IMAGE_BOOT_FILES ?
> >
> > To further explain the idea, I'm trying to use the new source plugin
> > and .wks.in to build a wic image that "mimic" a hddimg, where I can dd
> > the wic image to an USB drive and use that usb drive to install image
> > to my target machine (like hddimg did ).
> > first partition allow to boot into initramfs and it will execute the
> > install script. The script will then install system-boot
> > configuration, efi, kernel, rootfs.img and initrd from second
> > partition to target machine.
> >
> > My sample image here:
> >
> https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing
>
> OK, so some cleaned up snippets from the installer image recipe:
> # List of external things we must have had complete.
> do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete"
> do_rootfs[depends] += "full-image-to-install:do_image_complete"
> do_rootfs[depends] += "grub-efi:do_deploy"
> do_rootfs[depends] += "virtual/kernel:do_deploy"
>
> WKS_FILE = "sdimage-bootpart.wks"
>
> IMAGE_BOOT_FILES_append = "\
> ${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \
> core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \
> full-image-to-install-${MACHINE}.wic.xz;rootfs.img \
> grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \
> grub-installer.cfg;EFI/BOOT/grub.cfg \
> "
>
> And note that I reworked the installer for our case to write then update
> the real image's wic image rather than copy contents and then modify
> files due to other constraints (this project is also why I was pushing
> various changes to meta-secure-core).
>
> >
> >
> > On 5/16/19, 10:38 AM, "Tom Rini" wrote:
> >
> > On Thu, May 16, 2019 at 02:23:05AM +, Lee, Chee Yang wrote:
> > > Hi Tom,
> > >
> > > Would you mind to share with me how you make the installer image
> using wic?
> >
> > I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to
> populate
> > the partition as needed. I can't paste the exact image as it was
> done
> > for a customer.
> >
> > >
> > > Forwarded Message
> > > From: Tom Rini
> > > To: chee.yang@intel.com
> > > Cc: openembedded-core@lists.openembedded.org
> > > Subject: Re: [OE-core] [PATCH 0/4] add wic based image
> installer
> > > Date: Wed, 15 May 2019 11:33:43 -0400
> > >
> > > On Wed, May 08, 2019 at 02:40:58PM +0800,
> chee.yang@intel.com
> > > wrote:
> > >
> > > > From: Chee Yang Lee
> > > >
> > > > Existing installable image (hddimg/ISO) has limit of 4GB
> size, so to
> > > > build larger installable image, we can build it in wic
> based image.
> > > >
> > > > 2 partition needed for installable image
>
Re: [OE-core] [PATCH 0/4] add wic based image installer
Tom, Thanks for the sharing
I try out this method and it works
however I notice that sdimage-bootpart.wks format bootimg-partition with vfat
still limit the rootfs.img size to 4GB.
Anyway, with the similar trick and using a new .wks with combination of 2
partition below could build the installer image for image greater than 4GB.
- bootimg-efi with "wic: bootimg-efi: add label source parameter" patch
- bootimg-partition ( format in ext4 )
so I think it would be great to keep the enhancement for bootimg-efi while
ignore the other 3 patches.
On 5/16/19, 8:20 PM, "Tom Rini" wrote:
On Thu, May 16, 2019 at 11:48:02AM +, Lee, Chee Yang wrote:
> I couldn’t figure out how to make it with sdimage-bootpart.wks, can
> you tell me what did you set for IMAGE_BOOT_FILES ?
>
> To further explain the idea, I'm trying to use the new source plugin
> and .wks.in to build a wic image that "mimic" a hddimg, where I can dd
> the wic image to an USB drive and use that usb drive to install image
> to my target machine (like hddimg did ).
> first partition allow to boot into initramfs and it will execute the
> install script. The script will then install system-boot
> configuration, efi, kernel, rootfs.img and initrd from second
> partition to target machine.
>
> My sample image here:
>
https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing
OK, so some cleaned up snippets from the installer image recipe:
# List of external things we must have had complete.
do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete"
do_rootfs[depends] += "full-image-to-install:do_image_complete"
do_rootfs[depends] += "grub-efi:do_deploy"
do_rootfs[depends] += "virtual/kernel:do_deploy"
WKS_FILE = "sdimage-bootpart.wks"
IMAGE_BOOT_FILES_append = "\
${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \
core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \
full-image-to-install-${MACHINE}.wic.xz;rootfs.img \
grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \
grub-installer.cfg;EFI/BOOT/grub.cfg \
"
And note that I reworked the installer for our case to write then update
the real image's wic image rather than copy contents and then modify
files due to other constraints (this project is also why I was pushing
various changes to meta-secure-core).
>
>
> On 5/16/19, 10:38 AM, "Tom Rini" wrote:
>
> On Thu, May 16, 2019 at 02:23:05AM +, Lee, Chee Yang wrote:
> > Hi Tom,
> >
> > Would you mind to share with me how you make the installer image
using wic?
>
> I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate
> the partition as needed. I can't paste the exact image as it was done
> for a customer.
>
> >
> > ---- Forwarded Message ----
> > From: Tom Rini
> > To: chee.yang@intel.com
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer
> > Date: Wed, 15 May 2019 11:33:43 -0400
> >
> > On Wed, May 08, 2019 at 02:40:58PM +0800,
chee.yang@intel.com
> > wrote:
> >
> > > From: Chee Yang Lee
> > >
> > > Existing installable image (hddimg/ISO) has limit of 4GB
size, so to
> > > build larger installable image, we can build it in wic based
image.
> > >
> > > 2 partition needed for installable image
> > > - first partition build using source plugin bootimg-efi and
> > > configure to run install
> > > - second partition build with root.img, systemd-boot and
kernel to
> > > be install on target machine (this partition build using new
source
> > > plugin installer-partition)
> > >
> > > These patches:
> > > - add new source plugin for second partition
> > > - add new .wks
> > > - add new wic dependency and set default value for required
variable
> > > - allow source plugin bootimg-efi to configure to install
> > >
> > >
> > > To build the image, set WKS_FILE="wic-installer.wks.in" in
> >
Re: [OE-core] [PATCH 0/4] add wic based image installer
On Thu, May 16, 2019 at 11:48:02AM +, Lee, Chee Yang wrote:
> I couldn’t figure out how to make it with sdimage-bootpart.wks, can
> you tell me what did you set for IMAGE_BOOT_FILES ?
>
> To further explain the idea, I'm trying to use the new source plugin
> and .wks.in to build a wic image that "mimic" a hddimg, where I can dd
> the wic image to an USB drive and use that usb drive to install image
> to my target machine (like hddimg did ).
> first partition allow to boot into initramfs and it will execute the
> install script. The script will then install system-boot
> configuration, efi, kernel, rootfs.img and initrd from second
> partition to target machine.
>
> My sample image here:
> https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing
OK, so some cleaned up snippets from the installer image recipe:
# List of external things we must have had complete.
do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete"
do_rootfs[depends] += "full-image-to-install:do_image_complete"
do_rootfs[depends] += "grub-efi:do_deploy"
do_rootfs[depends] += "virtual/kernel:do_deploy"
WKS_FILE = "sdimage-bootpart.wks"
IMAGE_BOOT_FILES_append = "\
${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \
core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \
full-image-to-install-${MACHINE}.wic.xz;rootfs.img \
grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \
grub-installer.cfg;EFI/BOOT/grub.cfg \
"
And note that I reworked the installer for our case to write then update
the real image's wic image rather than copy contents and then modify
files due to other constraints (this project is also why I was pushing
various changes to meta-secure-core).
>
>
> On 5/16/19, 10:38 AM, "Tom Rini" wrote:
>
> On Thu, May 16, 2019 at 02:23:05AM +, Lee, Chee Yang wrote:
> > Hi Tom,
> >
> > Would you mind to share with me how you make the installer image using
> wic?
>
> I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate
> the partition as needed. I can't paste the exact image as it was done
> for a customer.
>
> >
> > -------- Forwarded Message ----
> > From: Tom Rini
> > To: chee.yang@intel.com
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer
> > Date: Wed, 15 May 2019 11:33:43 -0400
> >
> > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang@intel.com
> > wrote:
> >
> > > From: Chee Yang Lee
> > >
> > > Existing installable image (hddimg/ISO) has limit of 4GB size, so
> to
> > > build larger installable image, we can build it in wic based
> image.
> > >
> > > 2 partition needed for installable image
> > > - first partition build using source plugin bootimg-efi and
> > > configure to run install
> > > - second partition build with root.img, systemd-boot and kernel
> to
> > > be install on target machine (this partition build using new
> source
> > > plugin installer-partition)
> > >
> > > These patches:
> > > - add new source plugin for second partition
> > > - add new .wks
> > > - add new wic dependency and set default value for required
> variable
> > > - allow source plugin bootimg-efi to configure to install
> > >
> > >
> > > To build the image, set WKS_FILE="wic-installer.wks.in" in
> > > local.conf.
> >
> > I'm confused as to why we need most of these changes to support the
> end
> > goal. The enhancement to bootimg-efi makes conceptual sense. I
> don't
> > see why we need a new plugin to do the rest, however. What's being
> > done
> > here that we don't already do today? I say this having made
> installer
> > images for EFI machines using wic (in sumo, even). Thanks!
> >
> > --
> > Tom
> > --
> > ___
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> >
> >
>
> --
> Tom
>
>
--
Tom
signature.asc
Description: PGP signature
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/4] add wic based image installer
I couldn’t figure out how to make it with sdimage-bootpart.wks, can you tell me
what did you set for IMAGE_BOOT_FILES ?
To further explain the idea, I'm trying to use the new source plugin and
.wks.in to build a wic image that "mimic" a hddimg, where I can dd the wic
image to an USB drive and use that usb drive to install image to my target
machine (like hddimg did ).
first partition allow to boot into initramfs and it will execute the install
script. The script will then install system-boot configuration, efi, kernel,
rootfs.img and initrd from second partition to target machine.
My sample image here:
https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing
On 5/16/19, 10:38 AM, "Tom Rini" wrote:
On Thu, May 16, 2019 at 02:23:05AM +, Lee, Chee Yang wrote:
> Hi Tom,
>
> Would you mind to share with me how you make the installer image using
wic?
I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate
the partition as needed. I can't paste the exact image as it was done
for a customer.
>
> Forwarded Message
> From: Tom Rini
> To: chee.yang@intel.com
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer
> Date: Wed, 15 May 2019 11:33:43 -0400
>
> On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang@intel.com
> wrote:
>
> > From: Chee Yang Lee
> >
> > Existing installable image (hddimg/ISO) has limit of 4GB size, so to
> > build larger installable image, we can build it in wic based image.
> >
> > 2 partition needed for installable image
> > - first partition build using source plugin bootimg-efi and
> > configure to run install
> > - second partition build with root.img, systemd-boot and kernel to
> > be install on target machine (this partition build using new source
> > plugin installer-partition)
> >
> > These patches:
> > - add new source plugin for second partition
> > - add new .wks
> > - add new wic dependency and set default value for required
variable
> > - allow source plugin bootimg-efi to configure to install
> >
> >
> > To build the image, set WKS_FILE="wic-installer.wks.in" in
> > local.conf.
>
> I'm confused as to why we need most of these changes to support the
end
> goal. The enhancement to bootimg-efi makes conceptual sense. I don't
> see why we need a new plugin to do the rest, however. What's being
> done
> here that we don't already do today? I say this having made installer
> images for EFI machines using wic (in sumo, even). Thanks!
>
> --
> Tom
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
--
Tom
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/4] add wic based image installer
On Thu, May 16, 2019 at 02:23:05AM +, Lee, Chee Yang wrote: > Hi Tom, > > Would you mind to share with me how you make the installer image using wic? I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate the partition as needed. I can't paste the exact image as it was done for a customer. > > Forwarded Message > From: Tom Rini > To: chee.yang@intel.com > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > Date: Wed, 15 May 2019 11:33:43 -0400 > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang@intel.com > wrote: > > > From: Chee Yang Lee > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > build larger installable image, we can build it in wic based image. > > > > 2 partition needed for installable image > > - first partition build using source plugin bootimg-efi and > > configure to run install > > - second partition build with root.img, systemd-boot and kernel to > > be install on target machine (this partition build using new source > > plugin installer-partition) > > > > These patches: > > - add new source plugin for second partition > > - add new .wks > > - add new wic dependency and set default value for required variable > > - allow source plugin bootimg-efi to configure to install > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > local.conf. > > I'm confused as to why we need most of these changes to support the end > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > see why we need a new plugin to do the rest, however. What's being > done > here that we don't already do today? I say this having made installer > images for EFI machines using wic (in sumo, even). Thanks! > > -- > Tom > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > -- Tom signature.asc Description: PGP signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/4] add wic based image installer
Hi Tom, Would you mind to share with me how you make the installer image using wic? Forwarded Message From: Tom Rini To: chee.yang@intel.com Cc: openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer Date: Wed, 15 May 2019 11:33:43 -0400 On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang@intel.com wrote: > From: Chee Yang Lee > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > build larger installable image, we can build it in wic based image. > > 2 partition needed for installable image > - first partition build using source plugin bootimg-efi and > configure to run install > - second partition build with root.img, systemd-boot and kernel to > be install on target machine (this partition build using new source > plugin installer-partition) > > These patches: > - add new source plugin for second partition > - add new .wks > - add new wic dependency and set default value for required variable > - allow source plugin bootimg-efi to configure to install > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > local.conf. I'm confused as to why we need most of these changes to support the end goal. The enhancement to bootimg-efi makes conceptual sense. I don't see why we need a new plugin to do the rest, however. What's being done here that we don't already do today? I say this having made installer images for EFI machines using wic (in sumo, even). Thanks! -- Tom -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/4] add wic based image installer
Any comments on these patches? -Original Message- From: openembedded-core-boun...@lists.openembedded.org On Behalf Of chee.yang@intel.com Sent: Wednesday, May 8, 2019 2:41 PM To: openembedded-core@lists.openembedded.org Subject: [OE-core] [PATCH 0/4] add wic based image installer From: Chee Yang Lee Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image. 2 partition needed for installable image - first partition build using source plugin bootimg-efi and configure to run install - second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition) These patches: - add new source plugin for second partition - add new .wks - add new wic dependency and set default value for required variable - allow source plugin bootimg-efi to configure to install To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf. Chee Yang Lee (4): wic: add new installer-partition plugin wic: bootimg-efi: add label source parameter wic: add new wic-installer.wks for wic based image installer image_types_wic: add dependency for wic based image installer meta/classes/image_types_wic.bbclass | 6 + scripts/lib/wic/canned-wks/wic-installer.wks.in| 7 + scripts/lib/wic/plugins/source/bootimg-efi.py | 20 ++- .../lib/wic/plugins/source/installer-partition.py | 192 + 4 files changed, 220 insertions(+), 5 deletions(-) create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/4] add wic based image installer
On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang@intel.com wrote: > From: Chee Yang Lee > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to build > larger installable image, we can build it in wic based image. > > 2 partition needed for installable image > - first partition build using source plugin bootimg-efi and configure to run > install > - second partition build with root.img, systemd-boot and kernel to be > install on target machine (this partition build using new source plugin > installer-partition) > > These patches: > - add new source plugin for second partition > - add new .wks > - add new wic dependency and set default value for required variable > - allow source plugin bootimg-efi to configure to install > > > To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf. I'm confused as to why we need most of these changes to support the end goal. The enhancement to bootimg-efi makes conceptual sense. I don't see why we need a new plugin to do the rest, however. What's being done here that we don't already do today? I say this having made installer images for EFI machines using wic (in sumo, even). Thanks! -- Tom signature.asc Description: PGP signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/4] add wic based image installer
From: Chee Yang Lee Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image. 2 partition needed for installable image - first partition build using source plugin bootimg-efi and configure to run install - second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition) These patches: - add new source plugin for second partition - add new .wks - add new wic dependency and set default value for required variable - allow source plugin bootimg-efi to configure to install To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf. Chee Yang Lee (4): wic: add new installer-partition plugin wic: bootimg-efi: add label source parameter wic: add new wic-installer.wks for wic based image installer image_types_wic: add dependency for wic based image installer meta/classes/image_types_wic.bbclass | 6 + scripts/lib/wic/canned-wks/wic-installer.wks.in| 7 + scripts/lib/wic/plugins/source/bootimg-efi.py | 20 ++- .../lib/wic/plugins/source/installer-partition.py | 192 + 4 files changed, 220 insertions(+), 5 deletions(-) create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
