Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Feb 12, 2018 at 2:53 PM, John Baldwin  wrote:

> On Monday, February 12, 2018 02:31:46 PM Warner Losh wrote:
> > On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin  wrote:
> >
> > > On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote:
> > > > Greetings,
> > > >
> > > > As you may know, the Lua (http://www.lua.org) boot loader has been
> in
> > > the
> > > > works for some time. It started out life as a GSoC in 2014 by Pedro
> Souza
> > > > mentored by Wojciech A. Koszek. Rui Paulo created a svn project
> branch to
> > > > try to integrate it. I rebased that effort into a github branch which
> > > Pedro
> > > > Arthur fixed up. Over the past year, I've been cleaning up the boot
> > > loader
> > > > for other reasons, and found the time was ripe to start integrating
> this
> > > > into the tree. However, those integration efforts have taken a while
> as
> > > my
> > > > day-job work on the boot loader took priority. In the mean time, Ed
> Maste
> > > > and the FreeBSD Foundation funded Zakary Nafziger to enhance the
> original
> > > > GSoC Lua scripts to bring it closer to parity with the evolution of
> the
> > > > FORTH menu system since the GSoC project started.
> > > >
> > > > I'm pleased to announce that all these threads of development have
> > > > converged and I'll be pushing the FreeBSD Lua Loader later today.
> This
> > > > loader uses Lua as its scripting language instead of FORTH. While
> > > > co-existance is planned, the timeline for it is looking to be a few
> weeks
> > > > and I didn't want to delay pushing this into the tree for that.
> > > >
> > > > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > > > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do
> this,
> > > > you can do it in src/stand and install the result (be sure to have
> the
> > > > options for both the build and the install). This will replace your
> > > current
> > > > /boot/loader that is scripted with FORTH to one that's scripted with
> Lua.
> > > > It will install the lua scripts in /boot/lua. The boot is scripted
> with
> > > > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly
> advised
> > > > to create a backup copy of /boot/loader before testing (eg cp
> > > /boot/loader
> > > > /boot/loader_forth), since you'll need to boot that from boot2 if
> > > something
> > > > goes wrong. I've tested it extensively, though, with userboot.so and
> it's
> > > > test program, so all the initial kinks of finding the lua scripts,
> etc
> > > have
> > > > been worked out.
> > > >
> > > > While it's possible to build all the /boot/loader variants with Lua,
> I've
> > > > just tested a BIOS booting /boot/loader both with and without menus
> > > > enabled. I've not tested any of the other variants and the
> instructions
> > > for
> > > > testing some of them may be rather tedious (especially UEFI, if you
> want
> > > a
> > > > simple path to back out). Since there's not been full convergence
> > > testing,
> > > > you'll almost certainly find bumps in this system. Also, all the
> > > > build-system APIs are likely not yet final.
> > > >
> > > > I put  MFC after a month on the commit. Due to the heroic (dare I say
> > > > almost crazy) work of Kyle Evans on merging all the revs from
> -current to
> > > > 11, I'm planning a MFC to 11 after the co-existence issues are
> hammered
> > > > out. In 11, FORTH will be the default, and Lua will  be built by
> default,
> > > > but users will have to do something to use it. 12, both FORTH and Lua
> > > will
> > > > be built and installed, with Lua as default (barring unforeseen
> > > > complications). Once the co-existence stuff goes in, I imagine we'll
> make
> > > > the switch to Lua by default shortly after that. In 13, FORTH will be
> > > > removed unless there's a really really compelling case made to keep
> it.
> > > >
> > > > So please give it a spin and give me any feedback, documentation
> updates
> > > > and/or bug fixes. I'm especially interested in reviews from people
> that
> > > > have embedded Lua in other projects or experts in Lua that can
> improve
> > > the
> > > > robustness of the menu code.
> > >
> > > Do you have some memory usage numbers for LUA vs forth for the
> different
> > > BIOS loaders (text/data/bss sizes)?  For the EFI case we probably have
> lots
> > > of room, but for the non-EFI case we are constrained to 0xa -
> 0xa000
> > > for the text/data/bss and stack (in some cases like PXE booting the top
> > > can be lower than 0xa).  I'm not sure if we have any other
> platforms
> > > with similar memory constraints.
> > >
> >
> > Lua is about 60-70k larger than FORTH for this application:
> >
> > Forth based:
> > -r-xr-xr-x  1 root  wheel  385024 Feb 12 14:14
> > /home/imp/roots/amd64/boot/loader
> > -r-xr-xr-x  1 root  wheel  450560 Feb 12 14:14
> > /home/imp/roots/amd64/boot/zfsloader
> > Lua based:
> > -r-xr-xr-x  1 root  wheel  450560 Feb 12 14:20
> > 

Re: LUA boot loader coming very soon

[John Baldwin]

On Monday, February 12, 2018 02:31:46 PM Warner Losh wrote:
> On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin  wrote:
> 
> > On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote:
> > > Greetings,
> > >
> > > As you may know, the Lua (http://www.lua.org) boot loader has been in
> > the
> > > works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> > > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> > > try to integrate it. I rebased that effort into a github branch which
> > Pedro
> > > Arthur fixed up. Over the past year, I've been cleaning up the boot
> > loader
> > > for other reasons, and found the time was ripe to start integrating this
> > > into the tree. However, those integration efforts have taken a while as
> > my
> > > day-job work on the boot loader took priority. In the mean time, Ed Maste
> > > and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> > > GSoC Lua scripts to bring it closer to parity with the evolution of the
> > > FORTH menu system since the GSoC project started.
> > >
> > > I'm pleased to announce that all these threads of development have
> > > converged and I'll be pushing the FreeBSD Lua Loader later today. This
> > > loader uses Lua as its scripting language instead of FORTH. While
> > > co-existance is planned, the timeline for it is looking to be a few weeks
> > > and I didn't want to delay pushing this into the tree for that.
> > >
> > > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> > > you can do it in src/stand and install the result (be sure to have the
> > > options for both the build and the install). This will replace your
> > current
> > > /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> > > It will install the lua scripts in /boot/lua. The boot is scripted with
> > > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> > > to create a backup copy of /boot/loader before testing (eg cp
> > /boot/loader
> > > /boot/loader_forth), since you'll need to boot that from boot2 if
> > something
> > > goes wrong. I've tested it extensively, though, with userboot.so and it's
> > > test program, so all the initial kinks of finding the lua scripts, etc
> > have
> > > been worked out.
> > >
> > > While it's possible to build all the /boot/loader variants with Lua, I've
> > > just tested a BIOS booting /boot/loader both with and without menus
> > > enabled. I've not tested any of the other variants and the instructions
> > for
> > > testing some of them may be rather tedious (especially UEFI, if you want
> > a
> > > simple path to back out). Since there's not been full convergence
> > testing,
> > > you'll almost certainly find bumps in this system. Also, all the
> > > build-system APIs are likely not yet final.
> > >
> > > I put  MFC after a month on the commit. Due to the heroic (dare I say
> > > almost crazy) work of Kyle Evans on merging all the revs from -current to
> > > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> > > out. In 11, FORTH will be the default, and Lua will  be built by default,
> > > but users will have to do something to use it. 12, both FORTH and Lua
> > will
> > > be built and installed, with Lua as default (barring unforeseen
> > > complications). Once the co-existence stuff goes in, I imagine we'll make
> > > the switch to Lua by default shortly after that. In 13, FORTH will be
> > > removed unless there's a really really compelling case made to keep it.
> > >
> > > So please give it a spin and give me any feedback, documentation updates
> > > and/or bug fixes. I'm especially interested in reviews from people that
> > > have embedded Lua in other projects or experts in Lua that can improve
> > the
> > > robustness of the menu code.
> >
> > Do you have some memory usage numbers for LUA vs forth for the different
> > BIOS loaders (text/data/bss sizes)?  For the EFI case we probably have lots
> > of room, but for the non-EFI case we are constrained to 0xa - 0xa000
> > for the text/data/bss and stack (in some cases like PXE booting the top
> > can be lower than 0xa).  I'm not sure if we have any other platforms
> > with similar memory constraints.
> >
> 
> Lua is about 60-70k larger than FORTH for this application:
> 
> Forth based:
> -r-xr-xr-x  1 root  wheel  385024 Feb 12 14:14
> /home/imp/roots/amd64/boot/loader
> -r-xr-xr-x  1 root  wheel  450560 Feb 12 14:14
> /home/imp/roots/amd64/boot/zfsloader
> Lua based:
> -r-xr-xr-x  1 root  wheel  450560 Feb 12 14:20
> /home/imp/roots/amd64/boot/loader
> -r-xr-xr-x  1 root  wheel  520192 Feb 12 14:20
> /home/imp/roots/amd64/boot/zfsloader
> 
> So with Lua:
> %  size loader.bin
> textdata bss  dec   hex   filename
>   411840   22264   48304   482408   0x75c68   loader.bin
> % size zfsloader.bin
> textdata bss 

Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin  wrote:

> On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote:
> > Greetings,
> >
> > As you may know, the Lua (http://www.lua.org) boot loader has been in
> the
> > works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> > try to integrate it. I rebased that effort into a github branch which
> Pedro
> > Arthur fixed up. Over the past year, I've been cleaning up the boot
> loader
> > for other reasons, and found the time was ripe to start integrating this
> > into the tree. However, those integration efforts have taken a while as
> my
> > day-job work on the boot loader took priority. In the mean time, Ed Maste
> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> > GSoC Lua scripts to bring it closer to parity with the evolution of the
> > FORTH menu system since the GSoC project started.
> >
> > I'm pleased to announce that all these threads of development have
> > converged and I'll be pushing the FreeBSD Lua Loader later today. This
> > loader uses Lua as its scripting language instead of FORTH. While
> > co-existance is planned, the timeline for it is looking to be a few weeks
> > and I didn't want to delay pushing this into the tree for that.
> >
> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> > you can do it in src/stand and install the result (be sure to have the
> > options for both the build and the install). This will replace your
> current
> > /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> > It will install the lua scripts in /boot/lua. The boot is scripted with
> > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> > to create a backup copy of /boot/loader before testing (eg cp
> /boot/loader
> > /boot/loader_forth), since you'll need to boot that from boot2 if
> something
> > goes wrong. I've tested it extensively, though, with userboot.so and it's
> > test program, so all the initial kinks of finding the lua scripts, etc
> have
> > been worked out.
> >
> > While it's possible to build all the /boot/loader variants with Lua, I've
> > just tested a BIOS booting /boot/loader both with and without menus
> > enabled. I've not tested any of the other variants and the instructions
> for
> > testing some of them may be rather tedious (especially UEFI, if you want
> a
> > simple path to back out). Since there's not been full convergence
> testing,
> > you'll almost certainly find bumps in this system. Also, all the
> > build-system APIs are likely not yet final.
> >
> > I put  MFC after a month on the commit. Due to the heroic (dare I say
> > almost crazy) work of Kyle Evans on merging all the revs from -current to
> > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> > out. In 11, FORTH will be the default, and Lua will  be built by default,
> > but users will have to do something to use it. 12, both FORTH and Lua
> will
> > be built and installed, with Lua as default (barring unforeseen
> > complications). Once the co-existence stuff goes in, I imagine we'll make
> > the switch to Lua by default shortly after that. In 13, FORTH will be
> > removed unless there's a really really compelling case made to keep it.
> >
> > So please give it a spin and give me any feedback, documentation updates
> > and/or bug fixes. I'm especially interested in reviews from people that
> > have embedded Lua in other projects or experts in Lua that can improve
> the
> > robustness of the menu code.
>
> Do you have some memory usage numbers for LUA vs forth for the different
> BIOS loaders (text/data/bss sizes)?  For the EFI case we probably have lots
> of room, but for the non-EFI case we are constrained to 0xa - 0xa000
> for the text/data/bss and stack (in some cases like PXE booting the top
> can be lower than 0xa).  I'm not sure if we have any other platforms
> with similar memory constraints.
>

Lua is about 60-70k larger than FORTH for this application:

Forth based:
-r-xr-xr-x  1 root  wheel  385024 Feb 12 14:14
/home/imp/roots/amd64/boot/loader
-r-xr-xr-x  1 root  wheel  450560 Feb 12 14:14
/home/imp/roots/amd64/boot/zfsloader
Lua based:
-r-xr-xr-x  1 root  wheel  450560 Feb 12 14:20
/home/imp/roots/amd64/boot/loader
-r-xr-xr-x  1 root  wheel  520192 Feb 12 14:20
/home/imp/roots/amd64/boot/zfsloader

So with Lua:
%  size loader.bin
textdata bss  dec   hex   filename
  411840   22264   48304   482408   0x75c68   loader.bin
% size zfsloader.bin
textdata bss  dec   hex   filename
  478712   22968   52368   554048   0x87440   zfsloader.bin

so we're at 555k out of 640k before we look at heap usage. Of course, these
numbers are only going to get a lot worse as we pile-in new crypto
features... I know that 

Re: buildkernel with PORTS_MODULES fails: Variable OBJTOP is recursive

[Bryan Drewery]

On 2/12/2018 6:54 AM, Vladimir Zakharov wrote:
> Hello, Bryan!
> 
> On Fri, Feb 09, 2018, Bryan Drewery wrote:
>> On 2/1/2018 1:10 AM, Vladimir Zakharov wrote:
>>> Hello!
>>>
>>> For some time (about a week) building and installing kernel fails with
>>> the error "Variable OBJTOP is recursive." when going to build/install
>>> module from ports.
>>>
>>> Last successful build was at r328426. Next build at r328527 failed and
>>> still broken at r328649.
>>>
>>> Without PORTS_MODULES building and installing kernel succeeds. Another
>>> workaround: ignore error and build/install module directly from ports.
>>>
>>> # cat /etc/make.conf
>>> MALLOC_PRODUCTION=yes
>>> KERNCONF=GENERIC-NODEBUG GENERIC
>>> #KERNCONF= GENERIC-NODEBUG
>>> CPUTYPE?=native
>>> #PORTS_MODULES = graphics/drm-next-kmod emulators/virtualbox-ose-kmod
>>> PORTS_MODULES = graphics/drm-next-kmod 
>>>
>>> DOC_LANG = en_US.ISO8859-1 ru_RU.KOI8-R 
>>>
>>> WITH_DEBUG_PORTS = mail/neomutt
>>>
>>> WITH_CCACHE_BUILD=yes
>>> CCACHE_DIR=/home/ccache
>>>
>>> #DEVELOPER=yes
>>>
>>> ...
>>> Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel.full
>>> --- kernel.full ---
>>> linking kernel.full
>>> ctfmerge -L VERSION -g -o kernel.full ...
>>>   text  data   bssdec hex   filename
>>>   22584632   1376209   474   28709729   0x1b61361   kernel.full
>>> Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel.debug
>>> Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel
>>> --- all ---
>>> ===> Ports module graphics/drm-next-kmod (all)
>>> cd ${PORTSDIR:-/usr/ports}/graphics/drm-next-kmod; env  -u CC  -u CXX
>>> -u CPP  -u MAKESYSPATH  MAKEFLAGS="-j 4 -J 15,16 -j 4 -J 15,16 -D
>>> NO_MODULES_OBJ .MAKE.LEVEL.ENV=MAKELEVEL KERNEL=kernel TARGET=amd64
>>> TARGET_ARCH=amd64"  SYSDIR=/usr/src/sys
>>> PATH=
>>> SRC_BASE=/usr/src  OSVERSION=1200056
>>> WRKDIRPREFIX=/home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG make -B
>>> clean build
>>> ===>  Cleaning for drm-next-kmod-g20180117
>>> ===>  License BSD2CLAUSE MIT GPLv2 accepted by the user
>>> ===>   drm-next-kmod-g20180117 depends on file: /usr/local/sbin/pkg - found
>>> ===> Fetching all distfiles required by drm-next-kmod-g20180117 for building
>>> ===>  Extracting for drm-next-kmod-g20180117
>>> => SHA256 Checksum OK for 
>>> FreeBSDDesktop-kms-drm-g20180117-622fdd1_GH0.tar.gz.
>>> ===>  Patching for drm-next-kmod-g20180117
>>> ===>   drm-next-kmod-g20180117 depends on file: /usr/local/bin/ccache - 
>>> found
>>> ===>  Configuring for drm-next-kmod-g20180117
>>> ===>  Building for drm-next-kmod-g20180117
>>> ===> drm (all)
>>> Variable OBJTOP is recursive.
>>
>> For some reason I cannot recreate this issue.
> 
> It seems, setting WITH_AUTO_OBJ in /etc/src-env.conf causes an error.
> At least, removing it fixes build for me.
> 
> Build successful with the following settings:
> # cat /etc/src-env.conf
> WITH_META_MODE=
> #WITH_AUTO_OBJ=
> 
> # env | grep MAKE
> MAKEOBJDIRPREFIX=/home/obj
> 
> # cat /etc/make.conf
> MALLOC_PRODUCTION=yes
> KERNCONF= GENERIC-NODEBUG
> CPUTYPE?=native
> PORTS_MODULES= graphics/drm-next-kmod 
> DOC_LANG = en_US.ISO8859-1 ru_RU.KOI8-R 
> WITH_DEBUG_PORTS = mail/neomutt
> WITH_CCACHE_BUILD=yes
> CCACHE_DIR=/home/ccache
> DEVELOPER=yes
> 

Please try this patch (requires a buildkernel).

https://people.freebsd.org/~bdrewery/patches/kernel-portsmodules.diff

-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: posix_fallocate on ZFS

[John Baldwin]

On Saturday, February 10, 2018 01:46:33 PM Garrett Wollman wrote:
> In article
> ,
> asom...@freebsd.org writes:
> 
> >On Sat, Feb 10, 2018 at 10:28 AM, Willem Jan Withagen 
> >wrote:
> 
> >> Is there any expectation that this is going to fixed in any near future?
> 
> >No.  It's fundamentally impossible to support posix_fallocate on a COW
> >filesystem like ZFS.  Ceph should be taught to ignore an EINVAL result,
> >since the system call is merely advisory.
> 
> I don't think it's true that this is _fundamentally_ impossible.  What
> the standard requires would in essence be a per-object refreservation.
> ZFS supports refreservation, obviously, but not on a per-object basis.
> Furthermore, there are mechanisms to preallocate blocks for things
> like dumps.  So it *could* be done (as in, the concept is there), but
> it may not be practical.  (And ultimately, there are ways in which the
> administrator might manage the system that would defeat the desired
> effect, but that's out of the standard's scope.)  Given the semantic
> mismatch, though, I suspect it's unreasonable to expect anyone to
> prioritize implementation of such a feature.

I don't think posix_fallocate() can be compatible with COW.  Suppose you
do reserve a fixed set of blocks.  That ensures the first write has a
place to write, but not if you overwrite one of those blocks.  You'd have
to reserve another block to maintain the reservation each time you wrote
to a block, or you'd have to have a way to mark a file as not COW.  The
first case isn't really any better than not using posix_fallocate() in the
first place as you are still requiring writes to allocate blocks, and the
second seems a bit fraught with peril as well if the application is
expecting the non-COW'd file to be in sync with other files in the system
since presumably non-COW'd files couldn't be snapshotted, etc.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[John Baldwin]

On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote:
> Greetings,
> 
> As you may know, the Lua (http://www.lua.org) boot loader has been in the
> works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> try to integrate it. I rebased that effort into a github branch which Pedro
> Arthur fixed up. Over the past year, I've been cleaning up the boot loader
> for other reasons, and found the time was ripe to start integrating this
> into the tree. However, those integration efforts have taken a while as my
> day-job work on the boot loader took priority. In the mean time, Ed Maste
> and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> GSoC Lua scripts to bring it closer to parity with the evolution of the
> FORTH menu system since the GSoC project started.
> 
> I'm pleased to announce that all these threads of development have
> converged and I'll be pushing the FreeBSD Lua Loader later today. This
> loader uses Lua as its scripting language instead of FORTH. While
> co-existance is planned, the timeline for it is looking to be a few weeks
> and I didn't want to delay pushing this into the tree for that.
> 
> To try the loader, you'll need to build WITHOUT_FORTH=yes and
> WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> you can do it in src/stand and install the result (be sure to have the
> options for both the build and the install). This will replace your current
> /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> It will install the lua scripts in /boot/lua. The boot is scripted with
> /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> to create a backup copy of /boot/loader before testing (eg cp /boot/loader
> /boot/loader_forth), since you'll need to boot that from boot2 if something
> goes wrong. I've tested it extensively, though, with userboot.so and it's
> test program, so all the initial kinks of finding the lua scripts, etc have
> been worked out.
> 
> While it's possible to build all the /boot/loader variants with Lua, I've
> just tested a BIOS booting /boot/loader both with and without menus
> enabled. I've not tested any of the other variants and the instructions for
> testing some of them may be rather tedious (especially UEFI, if you want a
> simple path to back out). Since there's not been full convergence testing,
> you'll almost certainly find bumps in this system. Also, all the
> build-system APIs are likely not yet final.
> 
> I put  MFC after a month on the commit. Due to the heroic (dare I say
> almost crazy) work of Kyle Evans on merging all the revs from -current to
> 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> out. In 11, FORTH will be the default, and Lua will  be built by default,
> but users will have to do something to use it. 12, both FORTH and Lua will
> be built and installed, with Lua as default (barring unforeseen
> complications). Once the co-existence stuff goes in, I imagine we'll make
> the switch to Lua by default shortly after that. In 13, FORTH will be
> removed unless there's a really really compelling case made to keep it.
> 
> So please give it a spin and give me any feedback, documentation updates
> and/or bug fixes. I'm especially interested in reviews from people that
> have embedded Lua in other projects or experts in Lua that can improve the
> robustness of the menu code.

Do you have some memory usage numbers for LUA vs forth for the different
BIOS loaders (text/data/bss sizes)?  For the EFI case we probably have lots
of room, but for the non-EFI case we are constrained to 0xa - 0xa000
for the text/data/bss and stack (in some cases like PXE booting the top
can be lower than 0xa).  I'm not sure if we have any other platforms
with similar memory constraints.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Feb 12, 2018 at 9:51 AM, Warner Losh  wrote:

>
>
> On Mon, Feb 12, 2018 at 9:45 AM, Kris Moore  wrote:
>
>> On 02/12/2018 11:31, Warner Losh wrote:
>>
>>
>>
>> On Mon, Feb 12, 2018 at 9:26 AM, Kris Moore  wrote:
>>
>>> On 02/12/2018 10:27, Warner Losh wrote:
>>> > Greetings,
>>> >
>>> > As you may know, the Lua (http://www.lua.org) boot loader has been in
>>> the
>>> > works for some time. It started out life as a GSoC in 2014 by Pedro
>>> Souza
>>> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch
>>> to
>>> > try to integrate it. I rebased that effort into a github branch which
>>> Pedro
>>> > Arthur fixed up. Over the past year, I've been cleaning up the boot
>>> loader
>>> > for other reasons, and found the time was ripe to start integrating
>>> this
>>> > into the tree. However, those integration efforts have taken a while
>>> as my
>>> > day-job work on the boot loader took priority. In the mean time, Ed
>>> Maste
>>> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the
>>> original
>>> > GSoC Lua scripts to bring it closer to parity with the evolution of the
>>> > FORTH menu system since the GSoC project started.
>>> >
>>> > I'm pleased to announce that all these threads of development have
>>> > converged and I'll be pushing the FreeBSD Lua Loader later today. This
>>> > loader uses Lua as its scripting language instead of FORTH. While
>>> > co-existance is planned, the timeline for it is looking to be a few
>>> weeks
>>> > and I didn't want to delay pushing this into the tree for that.
>>> >
>>> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
>>> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do
>>> this,
>>> > you can do it in src/stand and install the result (be sure to have the
>>> > options for both the build and the install). This will replace your
>>> current
>>> > /boot/loader that is scripted with FORTH to one that's scripted with
>>> Lua.
>>> > It will install the lua scripts in /boot/lua. The boot is scripted with
>>> > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly
>>> advised
>>> > to create a backup copy of /boot/loader before testing (eg cp
>>> /boot/loader
>>> > /boot/loader_forth), since you'll need to boot that from boot2 if
>>> something
>>> > goes wrong. I've tested it extensively, though, with userboot.so and
>>> it's
>>> > test program, so all the initial kinks of finding the lua scripts, etc
>>> have
>>> > been worked out.
>>> >
>>> > While it's possible to build all the /boot/loader variants with Lua,
>>> I've
>>> > just tested a BIOS booting /boot/loader both with and without menus
>>> > enabled. I've not tested any of the other variants and the
>>> instructions for
>>> > testing some of them may be rather tedious (especially UEFI, if you
>>> want a
>>> > simple path to back out). Since there's not been full convergence
>>> testing,
>>> > you'll almost certainly find bumps in this system. Also, all the
>>> > build-system APIs are likely not yet final.
>>> >
>>> > I put  MFC after a month on the commit. Due to the heroic (dare I say
>>> > almost crazy) work of Kyle Evans on merging all the revs from -current
>>> to
>>> > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
>>> > out. In 11, FORTH will be the default, and Lua will  be built by
>>> default,
>>> > but users will have to do something to use it. 12, both FORTH and Lua
>>> will
>>> > be built and installed, with Lua as default (barring unforeseen
>>> > complications). Once the co-existence stuff goes in, I imagine we'll
>>> make
>>> > the switch to Lua by default shortly after that. In 13, FORTH will be
>>> > removed unless there's a really really compelling case made to keep it.
>>> >
>>> > So please give it a spin and give me any feedback, documentation
>>> updates
>>> > and/or bug fixes. I'm especially interested in reviews from people that
>>> > have embedded Lua in other projects or experts in Lua that can improve
>>> the
>>> > robustness of the menu code.
>>> >
>>> > Warner
>>> > ___
>>> > freebsd-current@freebsd.org mailing list
>>> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@f
>>> reebsd.org"
>>>
>>> Warner & Co,
>>>
>>> Great job all! Been wanting this for years, very excited to see it start
>>> landing :)
>>>
>>> One question, if we switch to Lua loader in TrueOS, do you know if the
>>> ZFS boot-environment menus have been implemented / tested yet?
>>
>>
>> I don't think that they have been implemented yet. I've certainly not
>> tested them.
>>
>> Warner
>>
>> Ok, I'll have to mess about with it. I recall there was some library
>> which created to expose ZFS in limited functionality over to the FORTH
>> code. That's probably a good place to start and see if that still works and
>> Lua can probe the pool 

Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Feb 12, 2018 at 9:45 AM, Kris Moore  wrote:

> On 02/12/2018 11:31, Warner Losh wrote:
>
>
>
> On Mon, Feb 12, 2018 at 9:26 AM, Kris Moore  wrote:
>
>> On 02/12/2018 10:27, Warner Losh wrote:
>> > Greetings,
>> >
>> > As you may know, the Lua (http://www.lua.org) boot loader has been in
>> the
>> > works for some time. It started out life as a GSoC in 2014 by Pedro
>> Souza
>> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch
>> to
>> > try to integrate it. I rebased that effort into a github branch which
>> Pedro
>> > Arthur fixed up. Over the past year, I've been cleaning up the boot
>> loader
>> > for other reasons, and found the time was ripe to start integrating this
>> > into the tree. However, those integration efforts have taken a while as
>> my
>> > day-job work on the boot loader took priority. In the mean time, Ed
>> Maste
>> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the
>> original
>> > GSoC Lua scripts to bring it closer to parity with the evolution of the
>> > FORTH menu system since the GSoC project started.
>> >
>> > I'm pleased to announce that all these threads of development have
>> > converged and I'll be pushing the FreeBSD Lua Loader later today. This
>> > loader uses Lua as its scripting language instead of FORTH. While
>> > co-existance is planned, the timeline for it is looking to be a few
>> weeks
>> > and I didn't want to delay pushing this into the tree for that.
>> >
>> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
>> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do
>> this,
>> > you can do it in src/stand and install the result (be sure to have the
>> > options for both the build and the install). This will replace your
>> current
>> > /boot/loader that is scripted with FORTH to one that's scripted with
>> Lua.
>> > It will install the lua scripts in /boot/lua. The boot is scripted with
>> > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly
>> advised
>> > to create a backup copy of /boot/loader before testing (eg cp
>> /boot/loader
>> > /boot/loader_forth), since you'll need to boot that from boot2 if
>> something
>> > goes wrong. I've tested it extensively, though, with userboot.so and
>> it's
>> > test program, so all the initial kinks of finding the lua scripts, etc
>> have
>> > been worked out.
>> >
>> > While it's possible to build all the /boot/loader variants with Lua,
>> I've
>> > just tested a BIOS booting /boot/loader both with and without menus
>> > enabled. I've not tested any of the other variants and the instructions
>> for
>> > testing some of them may be rather tedious (especially UEFI, if you
>> want a
>> > simple path to back out). Since there's not been full convergence
>> testing,
>> > you'll almost certainly find bumps in this system. Also, all the
>> > build-system APIs are likely not yet final.
>> >
>> > I put  MFC after a month on the commit. Due to the heroic (dare I say
>> > almost crazy) work of Kyle Evans on merging all the revs from -current
>> to
>> > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
>> > out. In 11, FORTH will be the default, and Lua will  be built by
>> default,
>> > but users will have to do something to use it. 12, both FORTH and Lua
>> will
>> > be built and installed, with Lua as default (barring unforeseen
>> > complications). Once the co-existence stuff goes in, I imagine we'll
>> make
>> > the switch to Lua by default shortly after that. In 13, FORTH will be
>> > removed unless there's a really really compelling case made to keep it.
>> >
>> > So please give it a spin and give me any feedback, documentation updates
>> > and/or bug fixes. I'm especially interested in reviews from people that
>> > have embedded Lua in other projects or experts in Lua that can improve
>> the
>> > robustness of the menu code.
>> >
>> > Warner
>> > ___
>> > freebsd-current@freebsd.org mailing list
>> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@f
>> reebsd.org"
>>
>> Warner & Co,
>>
>> Great job all! Been wanting this for years, very excited to see it start
>> landing :)
>>
>> One question, if we switch to Lua loader in TrueOS, do you know if the
>> ZFS boot-environment menus have been implemented / tested yet?
>
>
> I don't think that they have been implemented yet. I've certainly not
> tested them.
>
> Warner
>
> Ok, I'll have to mess about with it. I recall there was some library which
> created to expose ZFS in limited functionality over to the FORTH code.
> That's probably a good place to start and see if that still works and Lua
> can probe the pool for BE information.
>

OK. There's no such functionality in FORTH in the tree today. However, zfs
sets a number of env vars that loader.getenv() can get in lua.

Warner

Re: LUA boot loader coming very soon

[Kris Moore]

On 02/12/2018 11:31, Warner Losh wrote:
>
>
> On Mon, Feb 12, 2018 at 9:26 AM, Kris Moore  > wrote:
>
> On 02/12/2018 10:27, Warner Losh wrote:
> > Greetings,
> >
> > As you may know, the Lua (http://www.lua.org) boot loader has
> been in the
> > works for some time. It started out life as a GSoC in 2014 by
> Pedro Souza
> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project
> branch to
> > try to integrate it. I rebased that effort into a github branch
> which Pedro
> > Arthur fixed up. Over the past year, I've been cleaning up the
> boot loader
> > for other reasons, and found the time was ripe to start
> integrating this
> > into the tree. However, those integration efforts have taken a
> while as my
> > day-job work on the boot loader took priority. In the mean time,
> Ed Maste
> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the
> original
> > GSoC Lua scripts to bring it closer to parity with the evolution
> of the
> > FORTH menu system since the GSoC project started.
> >
> > I'm pleased to announce that all these threads of development have
> > converged and I'll be pushing the FreeBSD Lua Loader later
> today. This
> > loader uses Lua as its scripting language instead of FORTH. While
> > co-existance is planned, the timeline for it is looking to be a
> few weeks
> > and I didn't want to delay pushing this into the tree for that.
> >
> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to
> do this,
> > you can do it in src/stand and install the result (be sure to
> have the
> > options for both the build and the install). This will replace
> your current
> > /boot/loader that is scripted with FORTH to one that's scripted
> with Lua.
> > It will install the lua scripts in /boot/lua. The boot is
> scripted with
> > /boot/lua/loader.lua instead of /boot/loader.rc. You are
> strongly advised
> > to create a backup copy of /boot/loader before testing (eg cp
> /boot/loader
> > /boot/loader_forth), since you'll need to boot that from boot2
> if something
> > goes wrong. I've tested it extensively, though, with userboot.so
> and it's
> > test program, so all the initial kinks of finding the lua
> scripts, etc have
> > been worked out.
> >
> > While it's possible to build all the /boot/loader variants with
> Lua, I've
> > just tested a BIOS booting /boot/loader both with and without menus
> > enabled. I've not tested any of the other variants and the
> instructions for
> > testing some of them may be rather tedious (especially UEFI, if
> you want a
> > simple path to back out). Since there's not been full
> convergence testing,
> > you'll almost certainly find bumps in this system. Also, all the
> > build-system APIs are likely not yet final.
> >
> > I put  MFC after a month on the commit. Due to the heroic (dare
> I say
> > almost crazy) work of Kyle Evans on merging all the revs from
> -current to
> > 11, I'm planning a MFC to 11 after the co-existence issues are
> hammered
> > out. In 11, FORTH will be the default, and Lua will  be built by
> default,
> > but users will have to do something to use it. 12, both FORTH
> and Lua will
> > be built and installed, with Lua as default (barring unforeseen
> > complications). Once the co-existence stuff goes in, I imagine
> we'll make
> > the switch to Lua by default shortly after that. In 13, FORTH
> will be
> > removed unless there's a really really compelling case made to
> keep it.
> >
> > So please give it a spin and give me any feedback, documentation
> updates
> > and/or bug fixes. I'm especially interested in reviews from
> people that
> > have embedded Lua in other projects or experts in Lua that can
> improve the
> > robustness of the menu code.
> >
> > Warner
> > ___
> > freebsd-current@freebsd.org 
> mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> 
> > To unsubscribe, send any mail to
> "freebsd-current-unsubscr...@freebsd.org
> "
>
> Warner & Co,
>
> Great job all! Been wanting this for years, very excited to see it
> start
> landing :)
>
> One question, if we switch to Lua loader in TrueOS, do you know if the
> ZFS boot-environment menus have been implemented / tested yet?
>
>
> I don't think that they have been implemented yet. I've certainly not
> 

Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Feb 12, 2018 at 9:26 AM, Kris Moore  wrote:

> On 02/12/2018 10:27, Warner Losh wrote:
> > Greetings,
> >
> > As you may know, the Lua (http://www.lua.org) boot loader has been in
> the
> > works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> > try to integrate it. I rebased that effort into a github branch which
> Pedro
> > Arthur fixed up. Over the past year, I've been cleaning up the boot
> loader
> > for other reasons, and found the time was ripe to start integrating this
> > into the tree. However, those integration efforts have taken a while as
> my
> > day-job work on the boot loader took priority. In the mean time, Ed Maste
> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> > GSoC Lua scripts to bring it closer to parity with the evolution of the
> > FORTH menu system since the GSoC project started.
> >
> > I'm pleased to announce that all these threads of development have
> > converged and I'll be pushing the FreeBSD Lua Loader later today. This
> > loader uses Lua as its scripting language instead of FORTH. While
> > co-existance is planned, the timeline for it is looking to be a few weeks
> > and I didn't want to delay pushing this into the tree for that.
> >
> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> > you can do it in src/stand and install the result (be sure to have the
> > options for both the build and the install). This will replace your
> current
> > /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> > It will install the lua scripts in /boot/lua. The boot is scripted with
> > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> > to create a backup copy of /boot/loader before testing (eg cp
> /boot/loader
> > /boot/loader_forth), since you'll need to boot that from boot2 if
> something
> > goes wrong. I've tested it extensively, though, with userboot.so and it's
> > test program, so all the initial kinks of finding the lua scripts, etc
> have
> > been worked out.
> >
> > While it's possible to build all the /boot/loader variants with Lua, I've
> > just tested a BIOS booting /boot/loader both with and without menus
> > enabled. I've not tested any of the other variants and the instructions
> for
> > testing some of them may be rather tedious (especially UEFI, if you want
> a
> > simple path to back out). Since there's not been full convergence
> testing,
> > you'll almost certainly find bumps in this system. Also, all the
> > build-system APIs are likely not yet final.
> >
> > I put  MFC after a month on the commit. Due to the heroic (dare I say
> > almost crazy) work of Kyle Evans on merging all the revs from -current to
> > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> > out. In 11, FORTH will be the default, and Lua will  be built by default,
> > but users will have to do something to use it. 12, both FORTH and Lua
> will
> > be built and installed, with Lua as default (barring unforeseen
> > complications). Once the co-existence stuff goes in, I imagine we'll make
> > the switch to Lua by default shortly after that. In 13, FORTH will be
> > removed unless there's a really really compelling case made to keep it.
> >
> > So please give it a spin and give me any feedback, documentation updates
> > and/or bug fixes. I'm especially interested in reviews from people that
> > have embedded Lua in other projects or experts in Lua that can improve
> the
> > robustness of the menu code.
> >
> > Warner
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@
> freebsd.org"
>
> Warner & Co,
>
> Great job all! Been wanting this for years, very excited to see it start
> landing :)
>
> One question, if we switch to Lua loader in TrueOS, do you know if the
> ZFS boot-environment menus have been implemented / tested yet?


I don't think that they have been implemented yet. I've certainly not
tested them.

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Kris Moore]

On 02/12/2018 10:27, Warner Losh wrote:
> Greetings,
>
> As you may know, the Lua (http://www.lua.org) boot loader has been in the
> works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> try to integrate it. I rebased that effort into a github branch which Pedro
> Arthur fixed up. Over the past year, I've been cleaning up the boot loader
> for other reasons, and found the time was ripe to start integrating this
> into the tree. However, those integration efforts have taken a while as my
> day-job work on the boot loader took priority. In the mean time, Ed Maste
> and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> GSoC Lua scripts to bring it closer to parity with the evolution of the
> FORTH menu system since the GSoC project started.
>
> I'm pleased to announce that all these threads of development have
> converged and I'll be pushing the FreeBSD Lua Loader later today. This
> loader uses Lua as its scripting language instead of FORTH. While
> co-existance is planned, the timeline for it is looking to be a few weeks
> and I didn't want to delay pushing this into the tree for that.
>
> To try the loader, you'll need to build WITHOUT_FORTH=yes and
> WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> you can do it in src/stand and install the result (be sure to have the
> options for both the build and the install). This will replace your current
> /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> It will install the lua scripts in /boot/lua. The boot is scripted with
> /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> to create a backup copy of /boot/loader before testing (eg cp /boot/loader
> /boot/loader_forth), since you'll need to boot that from boot2 if something
> goes wrong. I've tested it extensively, though, with userboot.so and it's
> test program, so all the initial kinks of finding the lua scripts, etc have
> been worked out.
>
> While it's possible to build all the /boot/loader variants with Lua, I've
> just tested a BIOS booting /boot/loader both with and without menus
> enabled. I've not tested any of the other variants and the instructions for
> testing some of them may be rather tedious (especially UEFI, if you want a
> simple path to back out). Since there's not been full convergence testing,
> you'll almost certainly find bumps in this system. Also, all the
> build-system APIs are likely not yet final.
>
> I put  MFC after a month on the commit. Due to the heroic (dare I say
> almost crazy) work of Kyle Evans on merging all the revs from -current to
> 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> out. In 11, FORTH will be the default, and Lua will  be built by default,
> but users will have to do something to use it. 12, both FORTH and Lua will
> be built and installed, with Lua as default (barring unforeseen
> complications). Once the co-existence stuff goes in, I imagine we'll make
> the switch to Lua by default shortly after that. In 13, FORTH will be
> removed unless there's a really really compelling case made to keep it.
>
> So please give it a spin and give me any feedback, documentation updates
> and/or bug fixes. I'm especially interested in reviews from people that
> have embedded Lua in other projects or experts in Lua that can improve the
> robustness of the menu code.
>
> Warner
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Warner & Co,

Great job all! Been wanting this for years, very excited to see it start
landing :)

One question, if we switch to Lua loader in TrueOS, do you know if the
ZFS boot-environment menus have been implemented / tested yet?


-- 
Kris Moore
Vice President of Engineering
iXsystems
Enterprise Storage & Servers Driven By Open Source

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

LUA boot loader coming very soon

[Warner Losh]

Greetings,

As you may know, the Lua (http://www.lua.org) boot loader has been in the
works for some time. It started out life as a GSoC in 2014 by Pedro Souza
mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
try to integrate it. I rebased that effort into a github branch which Pedro
Arthur fixed up. Over the past year, I've been cleaning up the boot loader
for other reasons, and found the time was ripe to start integrating this
into the tree. However, those integration efforts have taken a while as my
day-job work on the boot loader took priority. In the mean time, Ed Maste
and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
GSoC Lua scripts to bring it closer to parity with the evolution of the
FORTH menu system since the GSoC project started.

I'm pleased to announce that all these threads of development have
converged and I'll be pushing the FreeBSD Lua Loader later today. This
loader uses Lua as its scripting language instead of FORTH. While
co-existance is planned, the timeline for it is looking to be a few weeks
and I didn't want to delay pushing this into the tree for that.

To try the loader, you'll need to build WITHOUT_FORTH=yes and
WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
you can do it in src/stand and install the result (be sure to have the
options for both the build and the install). This will replace your current
/boot/loader that is scripted with FORTH to one that's scripted with Lua.
It will install the lua scripts in /boot/lua. The boot is scripted with
/boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
to create a backup copy of /boot/loader before testing (eg cp /boot/loader
/boot/loader_forth), since you'll need to boot that from boot2 if something
goes wrong. I've tested it extensively, though, with userboot.so and it's
test program, so all the initial kinks of finding the lua scripts, etc have
been worked out.

While it's possible to build all the /boot/loader variants with Lua, I've
just tested a BIOS booting /boot/loader both with and without menus
enabled. I've not tested any of the other variants and the instructions for
testing some of them may be rather tedious (especially UEFI, if you want a
simple path to back out). Since there's not been full convergence testing,
you'll almost certainly find bumps in this system. Also, all the
build-system APIs are likely not yet final.

I put  MFC after a month on the commit. Due to the heroic (dare I say
almost crazy) work of Kyle Evans on merging all the revs from -current to
11, I'm planning a MFC to 11 after the co-existence issues are hammered
out. In 11, FORTH will be the default, and Lua will  be built by default,
but users will have to do something to use it. 12, both FORTH and Lua will
be built and installed, with Lua as default (barring unforeseen
complications). Once the co-existence stuff goes in, I imagine we'll make
the switch to Lua by default shortly after that. In 13, FORTH will be
removed unless there's a really really compelling case made to keep it.

So please give it a spin and give me any feedback, documentation updates
and/or bug fixes. I'm especially interested in reviews from people that
have embedded Lua in other projects or experts in Lua that can improve the
robustness of the menu code.

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildkernel with PORTS_MODULES fails: Variable OBJTOP is recursive

[Vladimir Zakharov]

Hello, Bryan!

On Fri, Feb 09, 2018, Bryan Drewery wrote:
> On 2/1/2018 1:10 AM, Vladimir Zakharov wrote:
> > Hello!
> > 
> > For some time (about a week) building and installing kernel fails with
> > the error "Variable OBJTOP is recursive." when going to build/install
> > module from ports.
> > 
> > Last successful build was at r328426. Next build at r328527 failed and
> > still broken at r328649.
> > 
> > Without PORTS_MODULES building and installing kernel succeeds. Another
> > workaround: ignore error and build/install module directly from ports.
> > 
> > # cat /etc/make.conf
> > MALLOC_PRODUCTION=yes
> > KERNCONF=GENERIC-NODEBUG GENERIC
> > #KERNCONF= GENERIC-NODEBUG
> > CPUTYPE?=native
> > #PORTS_MODULES = graphics/drm-next-kmod emulators/virtualbox-ose-kmod
> > PORTS_MODULES = graphics/drm-next-kmod 
> > 
> > DOC_LANG = en_US.ISO8859-1 ru_RU.KOI8-R 
> > 
> > WITH_DEBUG_PORTS = mail/neomutt
> > 
> > WITH_CCACHE_BUILD=yes
> > CCACHE_DIR=/home/ccache
> > 
> > #DEVELOPER=yes
> > 
> > ...
> > Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel.full
> > --- kernel.full ---
> > linking kernel.full
> > ctfmerge -L VERSION -g -o kernel.full ...
> >   text  data   bssdec hex   filename
> >   22584632   1376209   474   28709729   0x1b61361   kernel.full
> > Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel.debug
> > Building /home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG/kernel
> > --- all ---
> > ===> Ports module graphics/drm-next-kmod (all)
> > cd ${PORTSDIR:-/usr/ports}/graphics/drm-next-kmod; env  -u CC  -u CXX
> > -u CPP  -u MAKESYSPATH  MAKEFLAGS="-j 4 -J 15,16 -j 4 -J 15,16 -D
> > NO_MODULES_OBJ .MAKE.LEVEL.ENV=MAKELEVEL KERNEL=kernel TARGET=amd64
> > TARGET_ARCH=amd64"  SYSDIR=/usr/src/sys
> > PATH=
> > SRC_BASE=/usr/src  OSVERSION=1200056
> > WRKDIRPREFIX=/home/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG make -B
> > clean build
> > ===>  Cleaning for drm-next-kmod-g20180117
> > ===>  License BSD2CLAUSE MIT GPLv2 accepted by the user
> > ===>   drm-next-kmod-g20180117 depends on file: /usr/local/sbin/pkg - found
> > ===> Fetching all distfiles required by drm-next-kmod-g20180117 for building
> > ===>  Extracting for drm-next-kmod-g20180117
> > => SHA256 Checksum OK for 
> > FreeBSDDesktop-kms-drm-g20180117-622fdd1_GH0.tar.gz.
> > ===>  Patching for drm-next-kmod-g20180117
> > ===>   drm-next-kmod-g20180117 depends on file: /usr/local/bin/ccache - 
> > found
> > ===>  Configuring for drm-next-kmod-g20180117
> > ===>  Building for drm-next-kmod-g20180117
> > ===> drm (all)
> > Variable OBJTOP is recursive.
> 
> For some reason I cannot recreate this issue.

It seems, setting WITH_AUTO_OBJ in /etc/src-env.conf causes an error.
At least, removing it fixes build for me.

Build successful with the following settings:
# cat /etc/src-env.conf
WITH_META_MODE=
#WITH_AUTO_OBJ=

# env | grep MAKE
MAKEOBJDIRPREFIX=/home/obj

# cat /etc/make.conf
MALLOC_PRODUCTION=yes
KERNCONF= GENERIC-NODEBUG
CPUTYPE?=native
PORTS_MODULES= graphics/drm-next-kmod 
DOC_LANG = en_US.ISO8859-1 ru_RU.KOI8-R 
WITH_DEBUG_PORTS = mail/neomutt
WITH_CCACHE_BUILD=yes
CCACHE_DIR=/home/ccache
DEVELOPER=yes

-- 
Regards, | "In theory there is no difference between theory
  Vladimir Zakharov  | and practice. In practice there is."- Yogi Berra
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

[O. Hartmann]

On Mon, 12 Feb 2018 09:37:47 +0100
"O. Hartmann"  wrote:

> On Sat, 10 Feb 2018 11:52:18 +0100
> Olivier Cochard-Labbé  wrote:
> 
> > On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann  wrote:
> >   
> > >
> > > The moment any of the bridges gets an additional member epair interface
> > > (so the bridge
> > > has at least three members including the on reaching into the virtual
> > > router jail) the
> > > vbridge seems to operate unpredictable (to me). Pinging jails memeber of
> > > that vbridge
> > > are unreachable.
> > >
> > >
> > ​First idea:
> > Did you try with a more simple setup, like with just 3 jails members of one
> > bridge ?  
> > => I've tried it on a -head, and all 4 members  (3 jails and the host)
> > reach to communicate.
> > 
> > Second idea:
> > Can you check that all epairs have different MAC address each ?​
> > I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671  
> 
> Wow, that PR is from 2013(!) and it is still not solved?
> 
> > 
> > Regards,
> > 
> > Olivier
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"  
> 
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

After rebooting recent CURRENT, the view with "ifconfig -a ether" looked good
so far, each epair/bridge has its unique MAC.

But then, login on the jails and checking the epair's counterpart owned by the
VIMAGE jail, I found almost EVERY jail has the same MAC, even those jails
members of the same bridge:

[...]
jail 11  on bridge2
epair20129a: flags=8843 metric 0 mtu
1500 options=8
ether 02:68:d0:00:07:0a

jail 10 on bridge2
epair20128a: flags=8843 metric 0 mtu
1500 options=8
ether 02:68:d0:00:07:0a

jail 9 on bridge 1
epair10250a: flags=8843 metric 0 mtu
1500 options=8
ether 02:68:d0:00:07:0a

jail 8 on bridge1
epair10238a: flags=8843 metric 0 mtu
1500 options=8
ether 02:68:d0:00:07:0a

jail 7 on bridge0
epair238a: flags=8843 metric 0 mtu 1500
options=8
ether 02:68:d0:00:07:0a

epair251a: flags=8843 metric 0 mtu 1500
options=8
ether 02:68:d0:00:07:0a

The way I create epairs and put them into a jail's context/domain is straight
forward. In jail.conf, I have more generic setup with variables like:

# DNS Master
ns1 {
$if =   "2";
$ip4_addr = "10.10.0.${if}";
$ip4_cidr = "24";
$ip4_my_default_route = "10.10.0.1";
$vnet_if =  "epair${if}";
$home_bridge =  "${if_bridge_core}";

depend= "vrouter";

allow.raw_sockets=  "1";
}


and in the common portion of jail.conf definitions, I use this:

[...]

vnet =  "new";
vnet.interface ="${vnet_if}a";
persist;

exec.clean;

exec.start= "";
exec.start+="/sbin/ifconfig ${vnet_if}a inet
${ip4_addr}/${ip4_cidr} up"; exec.start+="/bin/sh /etc/rc";
exec.start+="/sbin/route add default ${ip4_my_default_route}";
exec.start+="/sbin/sysctl net.link.bridge.pfil_member=0";
exec.start+="/sbin/sysctl net.link.bridge.pfil_bridge=0";
exec.start+="/sbin/sysctl net.link.bridge.pfil_onlyip=0";

exec.stop=  "/bin/sh /etc/rc.shutdown";

exec.prestart=  "";
exec.prestart+= "ifconfig ${vnet_if} create";
exec.prestart+= "ifconfig ${vnet_if}b up";
exec.prestart+= "ifconfig ${home_bridge} addm ${vnet_if}b up";

exec.poststop=  "ifconfig ${home_bridge} deletem ${vnet_if}b";
exec.poststop+= "ifconfig ${vnet_if}b destroy";

exec.consolelog="/var/log/jail_${name}_console.log";


The big question here is: when a jail with VIMAGE kernel "swallows" a
epair-pseudo device, it leaves the ciontext or visibility of the host. How can
the FreeBSD VIMAGE kernel then know about what former epair's MAC was? Is this
mechanism maybe the culprit? It is just a thought, so I do not want to be
beheaded - I'm not much into system development.


Kind regards,

O. Hartmann
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Panic in prison_alloc() on boot

[Konstantin Belousov]

On Sun, Feb 11, 2018 at 11:50:43PM -0500, Ryan Stone wrote:
> I'm getting a persistent panic on boot in prison_allow().  The first
> case that I hit is this:
> 
> Fatal trap 12: page fault while in kernel mode
> cpuid = 10; apic id = 22
> fault virtual address   = 0x30
> fault code  = supervisor read data, page not present
> instruction pointer = 0x20:0x80ab9674
> stack pointer   = 0x28:0xfe00f8bb16f0
> frame pointer   = 0x28:0xfe00f8bb16f0
> code segment= base 0x0, limit 0xf, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags= interrupt enabled, resume, IOPL = 0
> current process = 90 (mount_nullfs)
> [ thread pid 90 tid 100913 ]
> Stopped at  prison_allow+0x4:   movqll+0xf(%rdi),%rax
> db> bt
> Tracing pid 90 tid 100913 td 0xf801a5aab000
> prison_allow() at prison_allow+0x4/frame 0xfe00f8bb16f0
Can you get the core dump and look at the arguments ?
It is most likely cred->cr_prison which is NULL.

> nullfs_mount() at nullfs_mount+0x31/frame 0xfe00f8bb1830
> vfs_donmount() at vfs_donmount+0x1415/frame 0xfe00f8bb1a80
> sys_nmount() at sys_nmount+0x72/frame 0xfe00f8bb1ac0
> amd64_syscall() at amd64_syscall+0xa48/frame 0xfe00f8bb1bf0
> fast_syscall_common() at fast_syscall_common+0x101/frame 0x7fffecb0
> 
> However if I comment out my nullfs mounts in /etc/fstab, I just get a
> panic in prison_allow() called from elsewhere.
> 
> I've seen this panic with r328936 (Feb 6), r329091 (Feb 9) and r329142 (Feb 
> 11)
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

[O. Hartmann]

On Sat, 10 Feb 2018 11:52:18 +0100
Olivier Cochard-Labbé  wrote:

> On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann  wrote:
> 
> >
> > The moment any of the bridges gets an additional member epair interface
> > (so the bridge
> > has at least three members including the on reaching into the virtual
> > router jail) the
> > vbridge seems to operate unpredictable (to me). Pinging jails memeber of
> > that vbridge
> > are unreachable.
> >
> >  
> ​First idea:
> Did you try with a more simple setup, like with just 3 jails members of one
> bridge ?
> => I've tried it on a -head, and all 4 members  (3 jails and the host)  
> reach to communicate.
> 
> Second idea:
> Can you check that all epairs have different MAC address each ?​
> I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Wow, that PR is from 2013(!) and it is still not solved?

> 
> Regards,
> 
> Olivier
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

[O. Hartmann]

On Sat, 10 Feb 2018 11:52:18 +0100
Olivier Cochard-Labbé  wrote:

> On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann  wrote:
> 
> >
> > The moment any of the bridges gets an additional member epair interface
> > (so the bridge
> > has at least three members including the on reaching into the virtual
> > router jail) the
> > vbridge seems to operate unpredictable (to me). Pinging jails memeber of
> > that vbridge
> > are unreachable.
> >
> >  
> ​First idea:
> Did you try with a more simple setup, like with just 3 jails members of one
> bridge ?
> => I've tried it on a -head, and all 4 members  (3 jails and the host)  
> reach to communicate.

Yes, I did.

I used to setup one bridge (bridge0) and three jails. Each jail owns its
epairXXa device with IP assigned. epairXXb of each jail is member of the
bridge. Bridge is up, epairXXb is up.

> 
> Second idea:
> Can you check that all epairs have different MAC address each ?​
> I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Very good shot! Thanks. No, they do not have all of them unique MAC adrresses
and in some occassions members of the very same bridge have the very same MAC
addresses, mostly the a-part of the epair:

From console:
[...]
ng_ether_ifnet_arrival_event: can't re-name node epair10250a
==>> epair20128a: Ethernet address: 02:ce:d0:00:07:0a
epair20128b: Ethernet address: 02:ce:d0:00:13:0b
epair20128a: link state changed to UP
epair20128b: link state changed to UP
epair20128b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20128a
==>> epair20129a: Ethernet address: 02:ce:d0:00:07:0a
epair20129b: Ethernet address: 02:ce:d0:00:14:0b
epair20129a: link state changed to UP
epair20129b: link state changed to UP
epair20129b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20129a

epair20XXX are member of bridge2 and dedicated epairs of jails.

The following is the desastrous picture of bridge1:

[...]
==>> epair235a: Ethernet address: 02:ce:d0:00:07:0a
epair235b: Ethernet address: 02:ce:d0:00:0d:0b
epair235a: link state changed to UP
epair235b: link state changed to UP
epair235b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair235a
==>> epair237a: Ethernet address: 02:ce:d0:00:07:0a
epair237b: Ethernet address: 02:ce:d0:00:0e:0b
epair237a: link state changed to UP
epair237b: link state changed to UP
epair237b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair237a
==>> epair251a: Ethernet address: 02:ce:d0:00:07:0a
epair251b: Ethernet address: 02:ce:d0:00:0f:0b
epair251a: link state changed to UP
epair251b: link state changed to UP
epair251b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair251a
==>> epair238a: Ethernet address: 02:ce:d0:00:07:0a
epair238b: Ethernet address: 02:ce:d0:00:10:0b
epair238a: link state changed to UP
epair238b: link state changed to UP
epair238b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair238a
[...]


This is on CURRENT (FreeBSD 12.0-CURRENT #36 r329150: Mon Feb 12 06:30:47 CET
2018 amd64).

I did check on Friday in the bureau and didn't catch it since I was checking on
each jail, but the console log accessed via dmesg revealed the problem very
easily.

I did a check on the 11.1-RELENG-p6 box and I got the same picture there,
different, but very similar setup. 

So I didn't see it in the masses of epairs our setup requires :-(

I'll go now for setting MAC addresses manually and check functionality again.

> 
> Regards,
> 
> Olivier

Kind regards,

Oliver
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"