Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Warner Losh
On Wed, Jun 17, 2020 at 9:30 PM Rodney W. Grimes <
freebsd-...@gndrsh.dnsmgr.net> wrote:

> > This is what we have running in AWS right now, kinda proof of concept but
> > it's not that difficult to generalize:
> >
> > [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mdconfig -lv
> > md0 preload   160M  -
> >
> > [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# df
> > Filesystem512-blocksUsed  Avail Capacity
> Mounted on
> > /dev/ufs/root_20200617071427 1300080 1220480  7960094%/
> > devfs  2   2  0   100%/dev
> > /dev/ufs/etc_20200617071427 99126384   273670%/etc
> > /dev/ufs/local_202006170714272746992 2572144 17484894%
> /usr/local
> > /dev/ufs/boot_20200617071427  389560  361208  2835293%/boot
> > tmpfs  65536 624  64912 1%/tmp
> > tmpfs  20480  16  20464 0%
> >  /usr/home/ssp-user
> > tmpfs 524288  336816 18747264%/var
> >
> > Root file system is untrimmed 1.2GB UFS, generated with mkuzip compressed
> > down to 160MB with the UZIP, and pre-loaded along with the kernel. The
> > /usr/local file system is read-only UFS+UZIP images placed directly onto
> > the GPT and probed out with GEOM_LABEL. Out of those only /etc is
> > read-write. The idea here is that the box should theoretically survive
> > total loss of connectivity to both root and the /usr/local storage (or we
> > can replace it on the fly with the new version).
> >
> > [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mount
> > /dev/ufs/root_20200617071427 on / (ufs, local, read-only)
> > devfs on /dev (devfs, local, multilabel)
> > /dev/ufs/etc_20200617071427 on /etc (ufs, local, synchronous)
> > /dev/ufs/local_20200617071427 on /usr/local (ufs, local, read-only)
> > /dev/ufs/boot_20200617071427 on /boot (ufs, local, read-only)
> > tmpfs on /tmp (tmpfs, local)
> > tmpfs on /usr/home/ssp-user (tmpfs, local)
> > tmpfs on /var (tmpfs, local)
> >
> > Configuration is dead simple:
> >
> > vfs.root.mountfrom="ufs:ufs/root_20200617071427"
> > image_load="YES"
> > image_name="/root.uzp"
> > image_type="mfs_root"
> > autoboot_delay="-1"
> >
> > It takes less than 100 lines of code I think to generate this out of
> > buildworld/buildkernel. 0 third party tools.
> >
> > Replace loading root from disk with loading it from HTTP server and it
> > would work just as good with the only need to load 1 or two files.
>
> I think your understating several of the stumbling blocks
> that exist here.  As Warner pointed out there are some
> pokey sticks around doing this over the net fs doing this
> from a local disk.
>

BTW, I think this is an excellent area to move from kernel compile options
to kernel env vars that maybe default to the kernel build options for
compat.

And, of course, the sharp bits would be less surprising if well
documented...

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> This is what we have running in AWS right now, kinda proof of concept but
> it's not that difficult to generalize:
> 
> [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mdconfig -lv
> md0 preload   160M  -
> 
> [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# df
> Filesystem512-blocksUsed  Avail Capacity  Mounted on
> /dev/ufs/root_20200617071427 1300080 1220480  7960094%/
> devfs  2   2  0   100%/dev
> /dev/ufs/etc_20200617071427 99126384   273670%/etc
> /dev/ufs/local_202006170714272746992 2572144 17484894%/usr/local
> /dev/ufs/boot_20200617071427  389560  361208  2835293%/boot
> tmpfs  65536 624  64912 1%/tmp
> tmpfs  20480  16  20464 0%
>  /usr/home/ssp-user
> tmpfs 524288  336816 18747264%/var
> 
> Root file system is untrimmed 1.2GB UFS, generated with mkuzip compressed
> down to 160MB with the UZIP, and pre-loaded along with the kernel. The
> /usr/local file system is read-only UFS+UZIP images placed directly onto
> the GPT and probed out with GEOM_LABEL. Out of those only /etc is
> read-write. The idea here is that the box should theoretically survive
> total loss of connectivity to both root and the /usr/local storage (or we
> can replace it on the fly with the new version).
> 
> [root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mount
> /dev/ufs/root_20200617071427 on / (ufs, local, read-only)
> devfs on /dev (devfs, local, multilabel)
> /dev/ufs/etc_20200617071427 on /etc (ufs, local, synchronous)
> /dev/ufs/local_20200617071427 on /usr/local (ufs, local, read-only)
> /dev/ufs/boot_20200617071427 on /boot (ufs, local, read-only)
> tmpfs on /tmp (tmpfs, local)
> tmpfs on /usr/home/ssp-user (tmpfs, local)
> tmpfs on /var (tmpfs, local)
> 
> Configuration is dead simple:
> 
> vfs.root.mountfrom="ufs:ufs/root_20200617071427"
> image_load="YES"
> image_name="/root.uzp"
> image_type="mfs_root"
> autoboot_delay="-1"
> 
> It takes less than 100 lines of code I think to generate this out of
> buildworld/buildkernel. 0 third party tools.
> 
> Replace loading root from disk with loading it from HTTP server and it
> would work just as good with the only need to load 1 or two files.

I think your understating several of the stumbling blocks
that exist here.  As Warner pointed out there are some
pokey sticks around doing this over the net fs doing this
from a local disk.

> 
> There is only one catch there - with real UEFI hardware sometimes there is
> small(ish) region followed by a hole and then the much bigger region.
> Unfortunately our loader picks smaller region for its work area and
> md_image loading mechanism is not good enough to either place it entirely
> into a bigger segment, or do scatter-gather and split it out and make the
> kernel do some VM trick to re-assemble later. But with some
> post-installworld cleaning if you can compress down the full image to some
> 30-40MB that usually works and has everything one may need including
> kitchen sink (i.e. python 3.7 with few modules). As I said, no woodo magic
> like famous crunchgen, just a very liberal application of the rm -rf.
> 
> With regards to ro vs. rw, recipy is "don't do it" :) If you want hard RO
> embedded into kernel - compress your image with mkuzip first and then embed
> it into kernel. This what FreeBSD/MIPS guys have mastered a lot (platform
> was very tight on flash), which is why geom_uzip is practically in every
> MIPS kernel config file. But that works (or should work) just as well on
> x64. Not only that would save lots of VM but also the RO proper attribute
> will be provided by the geom_uzip for free.
> 
> I am by the way hacking some way to populate /var with something more
> interesting that just stock /etc/mtree/BSD.var.dist, there will be a review
> request soon. :)

You may want to look at /etc/rc.initdiskless, the /var and /etc population
do have one existing solution.

> 
> -Max
> 
> On Wed, Jun 17, 2020 at 2:33 PM Miguel C  wrote:
> 
> > On Wed, Jun 17, 2020 at 9:28 PM Dave Cottlehuber 
> > wrote:
> >
> > > On Wed, 17 Jun 2020, at 17:52, Rodney W. Grimes wrote:
> > > > > Rodney W. Grimes  wrote:
> > > > > > > The "fake cd drive" is in the kernel, loader just copies the iso
> > > into
> > > > > > > memory like any other module, and by the time that's done you
> > just
> > > > > > > reboot into the newly installed system, which again uses
> > > > > > >
> > > > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > > > >   ^^^
> > > > > >
> > > > > > Argh, the cd9660 confused me, I think your doing a
> > > > > > "root on mfs/md"?
> > > > >
> > > > > loader.conf says
> > > > >
> > > > > rootfs_load="yes"
> > > > > rootfs_name="contents.izo"
> > > > > rootfs_type="md_image"
> > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > > >
> > > > > 

Re: Building modules gives error: "invalid output constraint '=@cce' in asm"

2020-06-17 Thread Rajesh Kumar
HI Andriy Gapon,

Thanks. It works.

On Wed, Jun 17, 2020 at 11:53 AM Andriy Gapon  wrote:

> On 17/06/2020 04:53, Rajesh Kumar wrote:
> > Then, I am trying to compile the driver modules and hit the
> > compilation error.  I haven't done "install world" as I don't want the
> base
> > 12.0 to be disturbed.
>
> You should do `make buildenv` and then do the module build in the subshell.
> This way you will be using a compiler (toolchain, in general) form the
> buildworld, not the currently installed compiler.
>
> --
> Andriy Gapon
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Simon J. Gerraty
Warner Losh  wrote:
> > loader.conf says
> >
> > rootfs_load="yes"
> > rootfs_name="contents.izo"
> > rootfs_type="md_image"
> > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> >
> > contents.izo is uzip'd contents.iso which file(1)
> > describes as ISO 9660 CD-ROM filesystem data ''
> >
> > That's for normal boot, for the loader 'install' command
> > it expects an uncompressed iso for rootfs.
> 
> Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> image to something that works with this.  Obviously we need a non-stock
> /boot/loader.conf file, or to type some commands manually at a loader
> prompt.  I believe the stock GENERIC kernel has the md_root support
> for this already, so it may not be that hard to do.

So obviously we don't use the GENERIC kernel, but
I don't think we have any magic except in 4th files and loader.conf
and for the loader install command its all in the loader itself,
and I've been keeping head up todate on recent fixes/improvements there
since for UEFI I'm using loader.efi built from head.

Oh and all the scripts run by init during boot are custom.

> Looking at the code, I think MD_ROOT alone is insufficient here...
> 
> If there's no MD root provided, we look for the symbols mfs_root and
> mfs_root_end, which I think means that rootfs_ in the above example
> needs to be md_root_ instead so that we find it.

FWIW our kernel options include

options CD9660
options MD_ROOT
options MD_ROOT_FSTYPE=\"cd9660\"

> You may need to have a custom kernel with 'options MD_ROOT_READONLY' because 
> isofs is read-only.
> 
> And there's a small chance you may need to define ROOTDEVNAME in the build as 
> well to be "cd9660:/dev/md0.uzip"

> Every time I do stuff like this I have to re-puzzle it out, alas, but
> these should give you some guide posts. It should be better documented
> in md(4), but isn't at the moment. 
> 
> I'd honestly try to get this setup working first loading all the files
> off a local disk before layering in the networking on top of that.

Agreed! Booting from say tftp://host/install.tar
is far more fragile - the tar file needs to present all the files in
correct order since we cannot seek backwards (much), and tftp sucks ;-)

userboot is very handy for testing all this stuff, though building it to
run on host (the way we do) seems broken in head.

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Maxim Sobolev
This is what we have running in AWS right now, kinda proof of concept but
it's not that difficult to generalize:

[root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mdconfig -lv
md0 preload   160M  -

[root@ip-172-31-10-188 /usr/local/etc/freeswitch]# df
Filesystem512-blocksUsed  Avail Capacity  Mounted on
/dev/ufs/root_20200617071427 1300080 1220480  7960094%/
devfs  2   2  0   100%/dev
/dev/ufs/etc_20200617071427 99126384   273670%/etc
/dev/ufs/local_202006170714272746992 2572144 17484894%/usr/local
/dev/ufs/boot_20200617071427  389560  361208  2835293%/boot
tmpfs  65536 624  64912 1%/tmp
tmpfs  20480  16  20464 0%
 /usr/home/ssp-user
tmpfs 524288  336816 18747264%/var

Root file system is untrimmed 1.2GB UFS, generated with mkuzip compressed
down to 160MB with the UZIP, and pre-loaded along with the kernel. The
/usr/local file system is read-only UFS+UZIP images placed directly onto
the GPT and probed out with GEOM_LABEL. Out of those only /etc is
read-write. The idea here is that the box should theoretically survive
total loss of connectivity to both root and the /usr/local storage (or we
can replace it on the fly with the new version).

[root@ip-172-31-10-188 /usr/local/etc/freeswitch]# mount
/dev/ufs/root_20200617071427 on / (ufs, local, read-only)
devfs on /dev (devfs, local, multilabel)
/dev/ufs/etc_20200617071427 on /etc (ufs, local, synchronous)
/dev/ufs/local_20200617071427 on /usr/local (ufs, local, read-only)
/dev/ufs/boot_20200617071427 on /boot (ufs, local, read-only)
tmpfs on /tmp (tmpfs, local)
tmpfs on /usr/home/ssp-user (tmpfs, local)
tmpfs on /var (tmpfs, local)

Configuration is dead simple:

vfs.root.mountfrom="ufs:ufs/root_20200617071427"
image_load="YES"
image_name="/root.uzp"
image_type="mfs_root"
autoboot_delay="-1"

It takes less than 100 lines of code I think to generate this out of
buildworld/buildkernel. 0 third party tools.

Replace loading root from disk with loading it from HTTP server and it
would work just as good with the only need to load 1 or two files.

There is only one catch there - with real UEFI hardware sometimes there is
small(ish) region followed by a hole and then the much bigger region.
Unfortunately our loader picks smaller region for its work area and
md_image loading mechanism is not good enough to either place it entirely
into a bigger segment, or do scatter-gather and split it out and make the
kernel do some VM trick to re-assemble later. But with some
post-installworld cleaning if you can compress down the full image to some
30-40MB that usually works and has everything one may need including
kitchen sink (i.e. python 3.7 with few modules). As I said, no woodo magic
like famous crunchgen, just a very liberal application of the rm -rf.

With regards to ro vs. rw, recipy is "don't do it" :) If you want hard RO
embedded into kernel - compress your image with mkuzip first and then embed
it into kernel. This what FreeBSD/MIPS guys have mastered a lot (platform
was very tight on flash), which is why geom_uzip is practically in every
MIPS kernel config file. But that works (or should work) just as well on
x64. Not only that would save lots of VM but also the RO proper attribute
will be provided by the geom_uzip for free.

I am by the way hacking some way to populate /var with something more
interesting that just stock /etc/mtree/BSD.var.dist, there will be a review
request soon. :)

-Max

On Wed, Jun 17, 2020 at 2:33 PM Miguel C  wrote:

> On Wed, Jun 17, 2020 at 9:28 PM Dave Cottlehuber 
> wrote:
>
> > On Wed, 17 Jun 2020, at 17:52, Rodney W. Grimes wrote:
> > > > Rodney W. Grimes  wrote:
> > > > > > The "fake cd drive" is in the kernel, loader just copies the iso
> > into
> > > > > > memory like any other module, and by the time that's done you
> just
> > > > > > reboot into the newly installed system, which again uses
> > > > > >
> > > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > > >   ^^^
> > > > >
> > > > > Argh, the cd9660 confused me, I think your doing a
> > > > > "root on mfs/md"?
> > > >
> > > > loader.conf says
> > > >
> > > > rootfs_load="yes"
> > > > rootfs_name="contents.izo"
> > > > rootfs_type="md_image"
> > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > >
> > > > contents.izo is uzip'd contents.iso which file(1)
> > > > describes as ISO 9660 CD-ROM filesystem data ''
> > > >
> > > > That's for normal boot, for the loader 'install' command
> > > > it expects an uncompressed iso for rootfs.
> > >
> > > Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> > > image to something that works with this.  Obviously we need a non-stock
> > > /boot/loader.conf file, or to type some commands manually at a loader
> > > prompt.  I believe the stock GENERIC 

Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Miguel C
On Wed, Jun 17, 2020 at 9:28 PM Dave Cottlehuber  wrote:

> On Wed, 17 Jun 2020, at 17:52, Rodney W. Grimes wrote:
> > > Rodney W. Grimes  wrote:
> > > > > The "fake cd drive" is in the kernel, loader just copies the iso
> into
> > > > > memory like any other module, and by the time that's done you just
> > > > > reboot into the newly installed system, which again uses
> > > > >
> > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > >   ^^^
> > > >
> > > > Argh, the cd9660 confused me, I think your doing a
> > > > "root on mfs/md"?
> > >
> > > loader.conf says
> > >
> > > rootfs_load="yes"
> > > rootfs_name="contents.izo"
> > > rootfs_type="md_image"
> > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > >
> > > contents.izo is uzip'd contents.iso which file(1)
> > > describes as ISO 9660 CD-ROM filesystem data ''
> > >
> > > That's for normal boot, for the loader 'install' command
> > > it expects an uncompressed iso for rootfs.
> >
> > Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> > image to something that works with this.  Obviously we need a non-stock
> > /boot/loader.conf file, or to type some commands manually at a loader
> > prompt.  I believe the stock GENERIC kernel has the md_root support
> > for this already, so it may not be that hard to do.
>
>
> Hi Miguel, all,
>
> I spent a bit of time on UEFI HTTP Boot earlier in the year in qemu,
> bhyve, and intel NUCs -- until everything in the world went to custard. I
> made some  rough notes[1] and I'll go through them again tonight with a
> fresh build. Hopefully its useful.
>
> What I got stuck on was the final pivot, I have never debugged this setup
> before and I'm still not clear at what point things fail. Olivier's PXE
> booting and BSDRP were a fantastic reference, and I assume they work in
> BSDRP already for him.
>
> Worth noting that LE TLS certs didn't play well with the PXE UEFI
> implementation on my intel NUC, this comes up as a very unhelpful error. At
> least use plain HTTP to get started.
>
> While my notes are amd64 oriented I'm very interested in using this for
> aarch64 locally & in the clowd.
>
> My loader.conf follows:
>
> boot_multicons="YES"
> console="efi,comconsole"
> comconsole_speed="115200"
> boot_verbose="YES"
> # make booting somewhat less painful
> #entropy_cache_load="NO"
> #kern.random.initial_seeding.bypass_before_seeding="0"
> # entropy_cache_load="YES"
> # boot_single="YES"
> tmpfs_load="YES"
> autoboot_delay="-1"
> # dump net vars
> # exec="show boot.netif.hwaddr"
> # exec="show boot.netif.ip"
> # exec="show boot.netif.netmask"
> # exec="show boot.netif.gateway"
> # ensure we have enough ram for our image
> vm.kmem_size=2G
> vfs.root.mountfrom="ufs:/dev/md0"
> # vfs.root.mountfrom.options=ro
> mfs_load="YES"
> mfs_type="md_image"
> mfs_name="/boot/mfs-miniroot"
>
> interesting these are different from what's above in the thread.
>
>
Ah thanks a lot for this and for the references, especially the first one
with all the notes :D

references:
>
> [1]: https://hackmd.io/@dch/H1X9RYEZr
> [mfsBSD]: https://mfsbsd.vx.sk/ still 150% awesome
> [olivier]:
> https://blog.cochard.me/2019/02/pxe-booting-of-freebsd-disk-image.html
> [BSDRP]: https://github.com/ocochard/BSDRP
>
> A+
> Dave
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Miguel C
On Wed, Jun 17, 2020 at 7:00 PM Rodney W. Grimes <
freebsd-...@gndrsh.dnsmgr.net> wrote:

> > On Tue., Jun. 16, 2020, 8:35 a.m. Rodney W. Grimes, <
> > freebsd-...@gndrsh.dnsmgr.net> wrote:
> >
> > > > I've been trying out FreeBSD with raspberry Pi4 (4GB) and wanted to
> see
> > > > what the state of HTTP BOOT is in FreeBSD, so I bumped into this!
> > > >
> > > > I'm curious if it should be possible to point to a img/iso directly
> (I
> > > > tried to use the img.xz unpacked it and make it available on a local
> web
> > > > server and that didn't seem to work for me)  but maybe thats cause
> those
> > > > images miss something, so arm64 aside does that work for amd64? I.E.
> > > using
> > > > the bootonly.iso?
> > >
> > > One problem you run into in attemtping this is even if you get an
> > > image downloaded and started that image is being provided by some
> > > memory device driver that emulates some type of iso device.
> > > FreeBSD does not have a driver for that device so once the kernel
> > > gets to the point of mounting its root file system it falls on
> > > its face with a mountroot failure.
> > >
> >
> > I donno what you are talking about Rodney, frankly! You information might
> > be way outdated, like 15 years outdated. :) FreeBSD comes with very
> decent
> > compressed image support in MD(4)+geom_uzip, which could be just UFS
> > snapshot or something created with mkimg utility. That said image could
> be
> > then either loaded after the kernel or embedded into one. Using this
> > approach we deploy our systems here, both kernel and all root + python
> > interpreter + custom gui installer fit into 40mb ISO but apart from
> loader
> > bits it's just two files.
>
> Max,
> Let me try explain what this user is actually experienceing,
> and that is taking a box stock linux distro, sticking it on a webserver
> and using PXE/HTTP booting to a running system.  FreeBSD can not
> achieve that today WITHOUT some additional work.  All that
> stuff like UFS snapshot, mkimg utlity, embeding the image is a ton
> of work compared to what others are doing.
>

Well actually I've been using opensbd to test not linux, I'm using
https://rpi4-uefi.dev/ (not the sotck firmware).

This firmware deos have UEFI HTTP support and as stated bellow UEFI 2.5
supports booting from just a iso or img file, ofc the OS needs to support
this too.

You can even point it to the internet FWIW but I was actually downloading
miniroot67.img  (openbsd installer) and serving it locally!

Creating a HTTP Boot entry manually in UEFI works perfect with this and I
can get to the shell, and even use it to install to a USB drive or sdcad.

>
> This person probably does not even have a running FreeBSD box to do
> any of your suggested solutions on.
>
>
No really I do have FreeBSD boxes, I just wanted to clarify that; I'm not
too familiar with the process of making system images but I can learn.


> So give me some credit, I have only been doing "diskless" since 1982,
> and actually do exactly what the OP is doing with Esxi, Ubuntu, Windows
> Installers, etc.  Just my choice of protocol at the PXE layer is NFS
> instead of HTTP, but my config files can do HTTP with 1 variable change
> and point a web server at the root of my boot images tree.
>
>
I even have a menu entry that sends me off to:
> https://netboot.xyz/

Regards,
> Rod
>
> >
> > -Max
> >
> >
> > > >
> > > > And on the other hand is there any doc on how to set up dhcp/http
> > > specific
> > > > to FreeBSD similar to
> https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup
> > > ?
> > >
> > > Since Linux uses this idea of a kernel payload and an initrd payload
> > > to boot with it is much easier to get these 2 things over the network
> > > and then have a workable system.  FreeBSD does not have the initrd
> > > payload and that complicates things, you need a functionaly filesystem
> > > avaliable at the end of kernel initilization.
> > > >
> > > > I looked into
> https://www.freebsd.org/doc/handbook/network-diskless.html
> > > > but that doesn't seem to be up to date (or at least it focuses only
> on
> > > PXE
> > > > and TFTP).
> > >
> > > Yes, old but workable.  I have a more advanced system that supports NFS
> > > booting using NFS support in PXE.  The only thing done via tftp is to
> > > upgrade the PXE running on the client to one that speaks NFS, then the
> > > kernel is loaded via NFS and the root file system is later provided
> > > via NFS.  The use of NFS provides very fast boots, and I do not need
> > > a web server to do it :-).
> > >
> > > > For clarification my ultimate goal is to use a few pi4's as "thin
> > > clients",
> > > > so eventually I will have to setup an image of the system with the
> needed
> > > > software (freerdp) but for starters I just wanted to check if
> pointing
> > > > directly to a img/iso would work and that does not seem to be the
> case.
> > >
> > > I would strongly suggest use of NFS instead of trying to provide an
> > > ISO image, 

Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rebecca Cran

On 6/17/20 2:11 PM, Rodney W. Grimes wrote:


Does freeBSD have any way to access these "Virtual Disk"
or Virtual CD images once we leave the world of the loader?
I believe we do not, as these are BIOS/UEFI devices that
require calls into the UEFI code, which, IIRC is gone
once we exit the loader and start the kernel proper,
or shortly there after.

As far as I know these devices well not be found by the FreeBSD
cam layer ATA or AHCI drivers as they do not present an actual
PCI device to find.


I just tried it on my Supermicro server and you're right, it doesn't 
show up under FreeBSD. I'm not sure if there's a way to access them via 
UEFI Runtime Services or if it's designed purely as a boot-time thing.



--
Rebecca Cran



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Dave Cottlehuber
On Wed, 17 Jun 2020, at 17:52, Rodney W. Grimes wrote:
> > Rodney W. Grimes  wrote:
> > > > The "fake cd drive" is in the kernel, loader just copies the iso into
> > > > memory like any other module, and by the time that's done you just
> > > > reboot into the newly installed system, which again uses
> > > >
> > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > >   ^^^
> > > 
> > > Argh, the cd9660 confused me, I think your doing a
> > > "root on mfs/md"?
> > 
> > loader.conf says
> > 
> > rootfs_load="yes"
> > rootfs_name="contents.izo"
> > rootfs_type="md_image"
> > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > 
> > contents.izo is uzip'd contents.iso which file(1)
> > describes as ISO 9660 CD-ROM filesystem data ''
> > 
> > That's for normal boot, for the loader 'install' command
> > it expects an uncompressed iso for rootfs.
> 
> Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> image to something that works with this.  Obviously we need a non-stock
> /boot/loader.conf file, or to type some commands manually at a loader
> prompt.  I believe the stock GENERIC kernel has the md_root support
> for this already, so it may not be that hard to do.


Hi Miguel, all,

I spent a bit of time on UEFI HTTP Boot earlier in the year in qemu, bhyve, and 
intel NUCs -- until everything in the world went to custard. I made some  rough 
notes[1] and I'll go through them again tonight with a fresh build. Hopefully 
its useful.

What I got stuck on was the final pivot, I have never debugged this setup 
before and I'm still not clear at what point things fail. Olivier's PXE booting 
and BSDRP were a fantastic reference, and I assume they work in BSDRP already 
for him.

Worth noting that LE TLS certs didn't play well with the PXE UEFI 
implementation on my intel NUC, this comes up as a very unhelpful error. At 
least use plain HTTP to get started.

While my notes are amd64 oriented I'm very interested in using this for aarch64 
locally & in the clowd.

My loader.conf follows:

boot_multicons="YES"
console="efi,comconsole"
comconsole_speed="115200"
boot_verbose="YES"
# make booting somewhat less painful
#entropy_cache_load="NO"
#kern.random.initial_seeding.bypass_before_seeding="0"
# entropy_cache_load="YES"
# boot_single="YES"
tmpfs_load="YES"
autoboot_delay="-1"
# dump net vars
# exec="show boot.netif.hwaddr"
# exec="show boot.netif.ip"
# exec="show boot.netif.netmask"
# exec="show boot.netif.gateway"
# ensure we have enough ram for our image
vm.kmem_size=2G
vfs.root.mountfrom="ufs:/dev/md0"
# vfs.root.mountfrom.options=ro
mfs_load="YES"
mfs_type="md_image"
mfs_name="/boot/mfs-miniroot"

interesting these are different from what's above in the thread.

references:

[1]: https://hackmd.io/@dch/H1X9RYEZr
[mfsBSD]: https://mfsbsd.vx.sk/ still 150% awesome
[olivier]: 
https://blog.cochard.me/2019/02/pxe-booting-of-freebsd-disk-image.html
[BSDRP]: https://github.com/ocochard/BSDRP

A+
Dave
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: magic file update?

2020-06-17 Thread Xin Li


On 6/17/20 8:08 AM, Antoine Brodin wrote:
> On Wed, Jun 17, 2020 at 4:00 PM Michael Butler
>  wrote:
>>
>> I'm seeing this message repeatedly during port builds. Should I be
>> concerned?
>>
>> file: File 5.39 supports only version 16 magic files.
>> `/usr/share/misc/magic.mgc' is version 14
> 
> Hi,
> 
> FILES and FILESDIR are overwritten in lib/libmagic/Makefile :-/

Thanks, this should have been fixed by r362279.  I should have used a
fresh installation for testing, sorry about that.

Cheers,



signature.asc
Description: OpenPGP digital signature


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> 
> > On Jun 17, 2020, at 12:12 PM, Warner Losh  wrote:
> > 
> > I missed the start of this thread, so maybe I'm missing a key detail. 
> > However, I thought UEFI didn't have a RAM-disk, per se, but that we could 
> > load memory areas and pass that into the kernel using freebsd-only methods. 
> > But UEFI is a bit weird, so maybe it will generate a virtual cdrom...
> 
> See 
> https://uefi.org/sites/default/files/resources/FINAL%20Pres4%20UEFI%20HTTP%20Boot.pdf
> 
> ? RAM Disk Standard
> 
> ? UEFI 2.5 defined RAM Disk device path nodes
> - Standard access to a RAM Disk in UEFI
> - Supports Virtual Disk and Virtual CD (ISO image) in persistent or
> volatile memory?

Does freeBSD have any way to access these "Virtual Disk"
or Virtual CD images once we leave the world of the loader?
I believe we do not, as these are BIOS/UEFI devices that
require calls into the UEFI code, which, IIRC is gone
once we exit the loader and start the kernel proper,
or shortly there after.

As far as I know these devices well not be found by the FreeBSD
cam layer ATA or AHCI drivers as they do not present an actual
PCI device to find.

> Rebecca Cran
-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rebecca Cran

> On Jun 17, 2020, at 12:12 PM, Warner Losh  wrote:
> 
> I missed the start of this thread, so maybe I'm missing a key detail. 
> However, I thought UEFI didn't have a RAM-disk, per se, but that we could 
> load memory areas and pass that into the kernel using freebsd-only methods. 
> But UEFI is a bit weird, so maybe it will generate a virtual cdrom...

See 
https://uefi.org/sites/default/files/resources/FINAL%20Pres4%20UEFI%20HTTP%20Boot.pdf

“ RAM Disk Standard

• UEFI 2.5 defined RAM Disk device path nodes
- Standard access to a RAM Disk in UEFI
- Supports Virtual Disk and Virtual CD (ISO image) in persistent or
volatile memory”


— 
Rebecca Cran
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Warner Losh
On Wed, Jun 17, 2020 at 12:56 PM Simon J. Gerraty  wrote:

> Warner Losh  wrote:
> > > loader.conf says
> > >
> > > rootfs_load="yes"
> > > rootfs_name="contents.izo"
> > > rootfs_type="md_image"
> > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > >
> > > contents.izo is uzip'd contents.iso which file(1)
> > > describes as ISO 9660 CD-ROM filesystem data ''
> > >
> > > That's for normal boot, for the loader 'install' command
> > > it expects an uncompressed iso for rootfs.
> >
> > Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> > image to something that works with this.  Obviously we need a non-stock
> > /boot/loader.conf file, or to type some commands manually at a loader
> > prompt.  I believe the stock GENERIC kernel has the md_root support
> > for this already, so it may not be that hard to do.
>
> So obviously we don't use the GENERIC kernel, but
> I don't think we have any magic except in 4th files and loader.conf
> and for the loader install command its all in the loader itself,
> and I've been keeping head up todate on recent fixes/improvements there
> since for UEFI I'm using loader.efi built from head.
>

Yea the loader will load the image, but the md driver looks for images of
type md_image or md_root and will create md devices for those devices it
finds.


> Oh and all the scripts run by init during boot are custom.
>
> > Looking at the code, I think MD_ROOT alone is insufficient here...
> >
> > If there's no MD root provided, we look for the symbols mfs_root and
> > mfs_root_end, which I think means that rootfs_ in the above example
> > needs to be md_root_ instead so that we find it.
>
> FWIW our kernel options include
>
> options CD9660
> options MD_ROOT
> options MD_ROOT_FSTYPE=\"cd9660\"
>

Yea, this lets you mout cd9660 images as root...

It also drives setting rootdevnames[0] which is used to create the default
mountroot script on loading the first md image.


> > You may need to have a custom kernel with 'options MD_ROOT_READONLY'
> because isofs is read-only.
> >
> > And there's a small chance you may need to define ROOTDEVNAME in the
> build as well to be "cd9660:/dev/md0.uzip"
>
> > Every time I do stuff like this I have to re-puzzle it out, alas, but
> > these should give you some guide posts. It should be better documented
> > in md(4), but isn't at the moment.
> >
> > I'd honestly try to get this setup working first loading all the files
> > off a local disk before layering in the networking on top of that.
>
> Agreed! Booting from say tftp://host/install.tar
> is far more fragile - the tar file needs to present all the files in
> correct order since we cannot seek backwards (much), and tftp sucks ;-)
>
> userboot is very handy for testing all this stuff, though building it to
> run on host (the way we do) seems broken in head.
>

Yea, of course. that makes sense. I'd forgotten userboot helps a lot. The
default build of it should be working, though...

Warner


> --sjg
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Warner Losh
On Wed, Jun 17, 2020 at 12:19 PM Rodney W. Grimes <
freebsd-...@gndrsh.dnsmgr.net> wrote:

> > On Wed, Jun 17, 2020 at 11:53 AM Rodney W. Grimes <
> > freebsd-...@gndrsh.dnsmgr.net> wrote:
> >
> > > > Rodney W. Grimes  wrote:
> > > > > > The "fake cd drive" is in the kernel, loader just copies the iso
> into
> > > > > > memory like any other module, and by the time that's done you
> just
> > > > > > reboot into the newly installed system, which again uses
> > > > > >
> > > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > > >   ^^^
> > > > >
> > > > > Argh, the cd9660 confused me, I think your doing a
> > > > > "root on mfs/md"?
> > > >
> > > > loader.conf says
> > > >
> > > > rootfs_load="yes"
> > > > rootfs_name="contents.izo"
> > > > rootfs_type="md_image"
> > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > >
> > > > contents.izo is uzip'd contents.iso which file(1)
> > > > describes as ISO 9660 CD-ROM filesystem data ''
> > > >
> > > > That's for normal boot, for the loader 'install' command
> > > > it expects an uncompressed iso for rootfs.
> > >
> > > Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> > > image to something that works with this.  Obviously we need a non-stock
> > > /boot/loader.conf file, or to type some commands manually at a loader
> > > prompt.  I believe the stock GENERIC kernel has the md_root support
> > > for this already, so it may not be that hard to do.
> > >
> >
> > Looking at the code, I think MD_ROOT alone is insufficient here...
>
> I was a bit worried about that, but hopefull.  We do work out of the
> box with a NFS root as long as the NIC is found during boot.  And
> given that I load the loader over NFS the loader can also find my
> /boot/ directory and the files in it, so that part is already solved.
>

Yes. If you NFS mount root, it works. But if you are loading a full image,
the boot loader has to do all that and pass the right meta-data to the
kernel for the kernel to know what to use for root.


> > If there's no MD root provided, we look for the symbols mfs_root and
> > mfs_root_end, which I think means that rootfs_ in the above example needs
> > to be md_root_ instead so that we find it.
>
> Isnt this all handled by the loader?  I think we have 2 slightly
> different cases here.  THe one sjg shows up, where you actually
> load the md_image from a  seperate file, and the case your talking
> about where you actually embed the kernel and image into a single
> file.
>

No. I'm talking about dynamically loading as well. The loader sets up the
symbols that point to the image, though I didn't thread my way all the way
through that. The loader sets it all up, and the kernel just uses the hints
the loader gives and the md driver does at least part of the heavy lifting
to make it happen.

>
> > You may need to have a custom kernel with 'options MD_ROOT_READONLY'
> > because isofs is read-only.
> >
> > And there's a small chance you may need to define ROOTDEVNAME in the
> build
> > as well to be "cd9660:/dev/md0.uzip"
>
> I do not think that is necessesary but I'll keep it in mind, at present
> I do over ride the vfs.root.mountfrom to point to my version specific
> root file system using some ipxe variables.
>

Usually that's enough, though when you want to use a MD device as root,
it's configured in weird ways that appear from the code to be driven by
ifdefs. And the default defines are for an uncompressed UFS filesystem
inside that MD image. Others are possible, but I believe require compile
time options.

> Every time I do stuff like this I have to re-puzzle it out, alas, but
> these
> > should give you some guide posts. It should be better documented in
> md(4),
> > but isn't at the moment.
>
> Thanks for exposing what may be some pointy sticks to stumble on.
>

Yea, There be a dragon or two here.

The ZRouter folks appear to have do a reroot to the
cdrom:9660:/dev/md0.uzip after dd'ing the installer image onto it  but
otherwise they don't reply on md stuff running direclty out of flash that's
been uncompressed. I'd thought sure they'd worked this out for network
booting, but if so it's not in the copy of the scripts I have.

> I'd honestly try to get this setup working first loading all the files off
> > a local disk before layering in the networking on top of that.
>
> Probalby a good idea, as the usually failure mode in the diskless
> word is black screens leaving little detail about what went wrong.
>

Yea. Having both serial and video consoles connected wouldn't hurt :)

Good Luck!

Warner


> > Warner
> --
> Rod Grimes
> rgri...@freebsd.org
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> On Wed, Jun 17, 2020 at 11:53 AM Rodney W. Grimes <
> freebsd-...@gndrsh.dnsmgr.net> wrote:
> 
> > > Rodney W. Grimes  wrote:
> > > > > The "fake cd drive" is in the kernel, loader just copies the iso into
> > > > > memory like any other module, and by the time that's done you just
> > > > > reboot into the newly installed system, which again uses
> > > > >
> > > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > > >   ^^^
> > > >
> > > > Argh, the cd9660 confused me, I think your doing a
> > > > "root on mfs/md"?
> > >
> > > loader.conf says
> > >
> > > rootfs_load="yes"
> > > rootfs_name="contents.izo"
> > > rootfs_type="md_image"
> > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > >
> > > contents.izo is uzip'd contents.iso which file(1)
> > > describes as ISO 9660 CD-ROM filesystem data ''
> > >
> > > That's for normal boot, for the loader 'install' command
> > > it expects an uncompressed iso for rootfs.
> >
> > Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> > image to something that works with this.  Obviously we need a non-stock
> > /boot/loader.conf file, or to type some commands manually at a loader
> > prompt.  I believe the stock GENERIC kernel has the md_root support
> > for this already, so it may not be that hard to do.
> >
> 
> Looking at the code, I think MD_ROOT alone is insufficient here...

I was a bit worried about that, but hopefull.  We do work out of the
box with a NFS root as long as the NIC is found during boot.  And
given that I load the loader over NFS the loader can also find my
/boot/ directory and the files in it, so that part is already solved.

> If there's no MD root provided, we look for the symbols mfs_root and
> mfs_root_end, which I think means that rootfs_ in the above example needs
> to be md_root_ instead so that we find it.

Isnt this all handled by the loader?  I think we have 2 slightly
different cases here.  THe one sjg shows up, where you actually
load the md_image from a  seperate file, and the case your talking
about where you actually embed the kernel and image into a single
file.

> 
> You may need to have a custom kernel with 'options MD_ROOT_READONLY'
> because isofs is read-only.
> 
> And there's a small chance you may need to define ROOTDEVNAME in the build
> as well to be "cd9660:/dev/md0.uzip"

I do not think that is necessesary but I'll keep it in mind, at present
I do over ride the vfs.root.mountfrom to point to my version specific
root file system using some ipxe variables.

> 
> Every time I do stuff like this I have to re-puzzle it out, alas, but these
> should give you some guide posts. It should be better documented in md(4),
> but isn't at the moment.

Thanks for exposing what may be some pointy sticks to stumble on.

> 
> I'd honestly try to get this setup working first loading all the files off
> a local disk before layering in the networking on top of that.

Probalby a good idea, as the usually failure mode in the diskless
word is black screens leaving little detail about what went wrong.

> Warner
-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Warner Losh
On Wed, Jun 17, 2020 at 12:06 PM Rebecca Cran  wrote:

>
> > On Jun 17, 2020, at 11:40 AM, Rodney W. Grimes <
> freebsd-...@gndrsh.dnsmgr.net> wrote:
> >
> > Does FreeBSD kernel have a driver that can talk to the UEFI ramdisk?
>
> I’m fairly sure UEFI generates it as a standard CD drive.
>

I missed the start of this thread, so maybe I'm missing a key detail.
However, I thought UEFI didn't have a RAM-disk, per se, but that we could
load memory areas and pass that into the kernel using freebsd-only methods.
But UEFI is a bit weird, so maybe it will generate a virtual cdrom...

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Warner Losh
On Wed, Jun 17, 2020 at 11:53 AM Rodney W. Grimes <
freebsd-...@gndrsh.dnsmgr.net> wrote:

> > Rodney W. Grimes  wrote:
> > > > The "fake cd drive" is in the kernel, loader just copies the iso into
> > > > memory like any other module, and by the time that's done you just
> > > > reboot into the newly installed system, which again uses
> > > >
> > > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> > >   ^^^
> > >
> > > Argh, the cd9660 confused me, I think your doing a
> > > "root on mfs/md"?
> >
> > loader.conf says
> >
> > rootfs_load="yes"
> > rootfs_name="contents.izo"
> > rootfs_type="md_image"
> > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> >
> > contents.izo is uzip'd contents.iso which file(1)
> > describes as ISO 9660 CD-ROM filesystem data ''
> >
> > That's for normal boot, for the loader 'install' command
> > it expects an uncompressed iso for rootfs.
>
> Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
> image to something that works with this.  Obviously we need a non-stock
> /boot/loader.conf file, or to type some commands manually at a loader
> prompt.  I believe the stock GENERIC kernel has the md_root support
> for this already, so it may not be that hard to do.
>

Looking at the code, I think MD_ROOT alone is insufficient here...

If there's no MD root provided, we look for the symbols mfs_root and
mfs_root_end, which I think means that rootfs_ in the above example needs
to be md_root_ instead so that we find it.

You may need to have a custom kernel with 'options MD_ROOT_READONLY'
because isofs is read-only.

And there's a small chance you may need to define ROOTDEVNAME in the build
as well to be "cd9660:/dev/md0.uzip"

Every time I do stuff like this I have to re-puzzle it out, alas, but these
should give you some guide posts. It should be better documented in md(4),
but isn't at the moment.

I'd honestly try to get this setup working first loading all the files off
a local disk before layering in the networking on top of that.

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rebecca Cran

> On Jun 17, 2020, at 11:40 AM, Rodney W. Grimes 
>  wrote:
> 
> Does FreeBSD kernel have a driver that can talk to the UEFI ramdisk?

I’m fairly sure UEFI generates it as a standard CD drive.

— 
Rebecca Cran

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> On Tue., Jun. 16, 2020, 8:35 a.m. Rodney W. Grimes, <
> freebsd-...@gndrsh.dnsmgr.net> wrote:
> 
> > > I've been trying out FreeBSD with raspberry Pi4 (4GB) and wanted to see
> > > what the state of HTTP BOOT is in FreeBSD, so I bumped into this!
> > >
> > > I'm curious if it should be possible to point to a img/iso directly (I
> > > tried to use the img.xz unpacked it and make it available on a local web
> > > server and that didn't seem to work for me)  but maybe thats cause those
> > > images miss something, so arm64 aside does that work for amd64? I.E.
> > using
> > > the bootonly.iso?
> >
> > One problem you run into in attemtping this is even if you get an
> > image downloaded and started that image is being provided by some
> > memory device driver that emulates some type of iso device.
> > FreeBSD does not have a driver for that device so once the kernel
> > gets to the point of mounting its root file system it falls on
> > its face with a mountroot failure.
> >
> 
> I donno what you are talking about Rodney, frankly! You information might
> be way outdated, like 15 years outdated. :) FreeBSD comes with very decent
> compressed image support in MD(4)+geom_uzip, which could be just UFS
> snapshot or something created with mkimg utility. That said image could be
> then either loaded after the kernel or embedded into one. Using this
> approach we deploy our systems here, both kernel and all root + python
> interpreter + custom gui installer fit into 40mb ISO but apart from loader
> bits it's just two files.

Max,
Let me try explain what this user is actually experienceing,
and that is taking a box stock linux distro, sticking it on a webserver
and using PXE/HTTP booting to a running system.  FreeBSD can not
achieve that today WITHOUT some additional work.  All that
stuff like UFS snapshot, mkimg utlity, embeding the image is a ton
of work compared to what others are doing.

This person probably does not even have a running FreeBSD box to do
any of your suggested solutions on.

So give me some credit, I have only been doing "diskless" since 1982,
and actually do exactly what the OP is doing with Esxi, Ubuntu, Windows
Installers, etc.  Just my choice of protocol at the PXE layer is NFS
instead of HTTP, but my config files can do HTTP with 1 variable change
and point a web server at the root of my boot images tree.

I even have a menu entry that sends me off to:
https://netboot.xyz/

Regards,
Rod

> 
> -Max
> 
> 
> > >
> > > And on the other hand is there any doc on how to set up dhcp/http
> > specific
> > > to FreeBSD similar to https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup
> > ?
> >
> > Since Linux uses this idea of a kernel payload and an initrd payload
> > to boot with it is much easier to get these 2 things over the network
> > and then have a workable system.  FreeBSD does not have the initrd
> > payload and that complicates things, you need a functionaly filesystem
> > avaliable at the end of kernel initilization.
> > >
> > > I looked into https://www.freebsd.org/doc/handbook/network-diskless.html
> > > but that doesn't seem to be up to date (or at least it focuses only on
> > PXE
> > > and TFTP).
> >
> > Yes, old but workable.  I have a more advanced system that supports NFS
> > booting using NFS support in PXE.  The only thing done via tftp is to
> > upgrade the PXE running on the client to one that speaks NFS, then the
> > kernel is loaded via NFS and the root file system is later provided
> > via NFS.  The use of NFS provides very fast boots, and I do not need
> > a web server to do it :-).
> >
> > > For clarification my ultimate goal is to use a few pi4's as "thin
> > clients",
> > > so eventually I will have to setup an image of the system with the needed
> > > software (freerdp) but for starters I just wanted to check if pointing
> > > directly to a img/iso would work and that does not seem to be the case.
> >
> > I would strongly suggest use of NFS instead of trying to provide an
> > ISO image, as you no longer need to store the ISO in memory on the
> > client box, and with a pi4 your already memory contrained.
> >
> > > Thanks.
> > > ___
> > > freebsd-current@freebsd.org mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > > To unsubscribe, send any mail to "
> > freebsd-current-unsubscr...@freebsd.org"
> > --
> > Rod Grimes
> > rgri...@freebsd.org
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> >
> >

-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


magic file update?

2020-06-17 Thread Michael Butler
I'm seeing this message repeatedly during port builds. Should I be
concerned?

file: File 5.39 supports only version 16 magic files.
`/usr/share/misc/magic.mgc' is version 14

imb
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> Rodney W. Grimes  wrote:
> > > The "fake cd drive" is in the kernel, loader just copies the iso into
> > > memory like any other module, and by the time that's done you just
> > > reboot into the newly installed system, which again uses
> > >
> > > vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> >   ^^^
> > 
> > Argh, the cd9660 confused me, I think your doing a
> > "root on mfs/md"?
> 
> loader.conf says
> 
> rootfs_load="yes"
> rootfs_name="contents.izo"
> rootfs_type="md_image"
> vfs.root.mountfrom="cd9660:/dev/md0.uzip"
> 
> contents.izo is uzip'd contents.iso which file(1)
> describes as ISO 9660 CD-ROM filesystem data ''
> 
> That's for normal boot, for the loader 'install' command
> it expects an uncompressed iso for rootfs.

Ok, now the puzzle is how much work to get from a stock FreeBSD .iso
image to something that works with this.  Obviously we need a non-stock
/boot/loader.conf file, or to type some commands manually at a loader
prompt.  I believe the stock GENERIC kernel has the md_root support
for this already, so it may not be that hard to do.


-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CTF: UEFI HTTP boot support

2020-06-17 Thread Rodney W. Grimes
> On 6/16/20 5:17 AM, Miguel C wrote:
> 
> > I've been trying out FreeBSD with raspberry Pi4 (4GB) and wanted to see
> > what the state of HTTP BOOT is in FreeBSD, so I bumped into this!
> >
> > I'm curious if it should be possible to point to a img/iso directly (I
> > tried to use the img.xz unpacked it and make it available on a local web
> > server and that didn't seem to work for me)  but maybe thats cause those
> > images miss something, so arm64 aside does that work for amd64? I.E. using
> > the bootonly.iso?
> 
> Unfortunately HTTP boot only works as far as the kernel: UEFI fetches 
> loader.efi, the loader fetches and runs the kernel over HTTP -- and then 
> you need to use NFS to mount the filesystem (or have a local root 
> filesystem).
> 
> 
> UEFI also has RamDisk support, but I don't think that's for remote 
> ISO/disk files, just local files.
> 

Does FreeBSD kernel have a driver that can talk to the UEFI ramdisk?

I believe that is how many of the other distro's
are actually able to do this "load the stock .iso over the
network and just run from that in memory."  I have booted
serveral things this way and it is a nice feature.

> -- 
> Rebecca Cran
-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: magic file update?

2020-06-17 Thread Antoine Brodin
On Wed, Jun 17, 2020 at 4:00 PM Michael Butler
 wrote:
>
> I'm seeing this message repeatedly during port builds. Should I be
> concerned?
>
> file: File 5.39 supports only version 16 magic files.
> `/usr/share/misc/magic.mgc' is version 14

Hi,

FILES and FILESDIR are overwritten in lib/libmagic/Makefile :-/

Antoine
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Building modules gives error: "invalid output constraint '=@cce' in asm"

2020-06-17 Thread Andriy Gapon
On 17/06/2020 04:53, Rajesh Kumar wrote:
> Then, I am trying to compile the driver modules and hit the
> compilation error.  I haven't done "install world" as I don't want the base
> 12.0 to be disturbed.

You should do `make buildenv` and then do the module build in the subshell.
This way you will be using a compiler (toolchain, in general) form the
buildworld, not the currently installed compiler.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"