Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

[John Baldwin]

On 8/20/18 9:00 PM, O. Hartmann wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> Am Mon, 20 Aug 2018 21:24:21 +0200
> "O. Hartmann"  schrieb:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA512
>>
>> Building NanoBSD world on CURRENT r338113 fails due to:
>>
>> [...]
>> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&  MK_TESTS=no
>> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
>> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE 
>> CRUNCH_CFLAGS=-DRESCUE
>> MK_AUTO_OBJ=no   obj make[5]: 
>> "/pool/sources/CURRENT/src/tools/build/mk/Makefile.boot"
>> line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.  Copy 
>> the header to
>> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused by 
>> Makefile
>> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error code 1
>>
>> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
>> [...]
>>
>>
>> This problem occured during today's source updates since I was able to build 
>> the NanoBSD
>> image I intend to build yesterday ~ r338060.
>>
>> What is going wrong?
> 
> It seems the problem has been introduced after r338095, since r338095 builds 
> ok, while
> r338096 doesn't.

338096 added this check to catch a kind of error in our Makefiles.  Alex (cc'd) 
can
help with figuring out what the error is.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Mark Millard]

When I discovered that you were building debug versions
of devel/llvm60 that also meant that my Pine64+ 2GB
specifics do not make a useful comparison (do not
show how small an environment can be for a build).

Last I tried such a debug build was on a powerpc64
with 16 MiBytes RAM and it took having a swap space
of 10 MiBytes (9 insufficient) but it was devel/llvm40
and the machine was not in use for anything else
significant and was using UFS, unlike your more
involved context (ZFS, large applications running).

For your usage, it is not clear to me that 32 GiBytes
is all that much RAM, nor that 32GiByte RAM + 2 GiByte
swap is all that big of a total. To me it seemed odd
that your swap was so small for what you described as
running in your environment vs. the amount of RAM.

If it turns out handy, the stress test (scaled to 32
GiBytes) should show how your context handles low free
RAM over extended periods. (Without the complications
or delays of a build being involved.) (The Mark Johnston
patches expose more than just any "was killed" notice.)

My expectation is that the stress test would over time
show OOM kills even if you expanded the swap space so
that it clearly had notable space not in use at the
time.

It may be that setting vm.pageout_oom_seq large enough
would help your builds complete by tolerating low
free RAM for a longer time.

Something I only just learned was that by default lld
is multi-threaded. I've been told that this mode of
operation uses more RAM but I've no knowledge of my
own about the tradeoffs. They might be bigger for
debug builds than for non-debug builds. Doing a quick
experiment showed lld using 5 threads or so unless
I used -Wl,--no-threads . (I was using the cc/c++
command interface.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Newly upgraded -CURRENT box does not boot

[Kyle Evans]

On Mon, Aug 20, 2018 at 4:27 PM, Brett Gmoser
 wrote:
> Hi there,
>
> I was told to e-mail these addresses with this.
>
> I did an `svn update` on /usr/src last night, build world and kernel as
> usual. This morning I installed the kernel, booted into single user,
> installed world and did mergemaster -Ui as usual. The new kernel had booted
> fine. Upon reboot, the machine will no longer boot:
>
> Startup error in /boot/lua/loader.lua:
> LUA ERROR: cannot open /boot/lua/loader.lua: no such file or directory
>
> can't load 'kernel'
>
> Many things in the bootloader do not work, including "boot kernel.old", "ls
> /boot", and various other things (most if not all just result in "Command
> failed"). Interestingly, "ls /mnt" works, other directories do not. That's
> the only clue I have.
>
> I'm able to reboot in an installer image and mount the drive just fine.
> Everything is there and is as expected, including /boot/lua/loader.lua.
>
> I re-installed everything in /usr/src/stand (chroot'd on the installer
> image, and "cd /usr/src/stand && make clean all install"). This did not fix
> the problem.
>
> Does anybody happen to have any ideas?
>

To briefly follow up and summarize the current standing here following
some more discussion/attempts to fix on IRC:

1.) x86 BIOS boot
2.) Problem appears for both forthloader and lualoader
3.) Early March loader works, recent loader does not [Only tried
loader from the past ~day]
4.) ls / works, ls /mnt works, ls /boot and other directories fails
5.) However, /boot is confirmed intact and populated by booting via
11.2 install media and inspecting local disk

We'll hopefully be having a bisect session tomorrow to figure out
where exactly this broke so that maybe Brett has a chance to upgrade
to 12.0, unless this sounds familiar to someone and the cause is
obvious. =)

Thanks,

Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Rodney W. Grimes]

> On Mon, Aug 20, 2018 at 11:04 PM Mark Millard  wrote:
> 
> > Rodney W. Grimes freebsd-rwg at pdx.rh.CN85.dnsmgr.net wrote on
> > Mon Aug 20 14:26:55 UTC 2018 :
> >
> > > > On 20 Aug 2018, at 05:01, blubee blubeeme 
> > wrote:
> > > > >
> > > > > I am running current compiling LLVM60 and when it comes to linking
> > > > > basically all the processes on my computer gets killed; Chrome,
> > Firefox and
> > > > > some of the LLVM threads as well
> > > > ...
> > > > > llvm/build % ninja -j8
> > > > > [2408/2473] Building CXX object
> > > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > > > FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > > > /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> > > > > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes
> > -I../lib/Passes
> > > > > -Iinclude -I../include -isystem /usr/local/include -fPIC
> > > > > -fvisibility-inlines-hidden -Werror=date-time
> > > > > -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> > > > > -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> > > > > -Wmissing-field-initializers -pedantic -Wno-long-long
> > > > > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> > > > > -Wstring-conversion -fdiagnostics-color -g-fno-exceptions
> > -fno-rtti -MD
> > > > > -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> > > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> > > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> > > > > ../lib/Passes/PassBuilder.cpp
> > > > > c++: error: unable to execute command: Killed
> > > >
> > > > It is running out of RAM while running multiple parallel link jobs.  If
> > > > you are building using WITH_DEBUG, turn that off, it consumes large
> > > > amounts of memory.  If you must have debug info, try adding the
> > > > following flag to the CMake command line:
> > > >
> > > > -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> > > >
> > > > That will limit the amount of parallel link jobs to 1, even if you
> > > > specify -j 8 to gmake or ninja.
> > > >
> > > > Brooks, it would not be a bad idea to always use this CMake flag in the
> > > > llvm ports. :)
> > >
> > > And this may also fix the issues that all the small
> > > memory (aka, RPI*) buliders are facing when trying
> > > to do -j4?
> >
> > It may help, but:
> >
> > Even just compiles with no links running can get the kills
> > in such small system contexts.
> >
> > And going for a simpler context that can demonstrate
> > the behavior . . .
> >
> > Taking a Pine64+ 2GB as an example (4 cores with 1
> > HW-thread per core, 2 GiBytes of RAM, USB device for
> > root file system and swap partition):
> >
> > In another login:
> > # stress -d 2 -m 4 --vm-keep --vm-bytes 536870912
> >
> > That "4" and "536870912" total to the 2 GiBytes so
> > swapping is induced for the context in question.
> > (Scale --vm-bytes appropriately to context.)
> > [Note: I had 3 GiBytes of swap space in a partition
> > for the below.]
> >
> > [stress is from the port sysutils/stress .]
> >
> > I had left the default vm.pageout_oom_seq=12 in place for this,
> > making the kills easier to get than the 120 figure would. It
> > does not take very long generally for some sort of message to
> > show up. Sometimes kills happen:
> >
> > My test environment has Mark Johnston patches to report
> > things not normally reported:
> >
> > waited 9s for async swap write
> > waited 9s for swap buffer
> > waited 9s for async swap write
> > waited 9s for async swap write
> > waited 9s for async swap write
> > v_free_count: 1357, v_inactive_count: 1
> > Aug 20 06:04:27 pine64 kernel: pid 1010 (stress), uid 0, was killed: out
> > of swap space
> > waited 5s for async swap write
> > waited 5s for swap buffer
> > waited 5s for async swap write
> > waited 5s for async swap write
> > waited 5s for async swap write
> > waited 13s for async swap write
> > waited 12s for swap buffer
> > waited 13s for async swap write
> > waited 12s for async swap write
> > waited 12s for async swap write
> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 161177, size: 131072
> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164766, size: 65536
> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164064, size: 12288
> > . . .
> >
> > (I made multiple runs, most manually stopped. None run for
> > all that long.)
> >
> > (Warning: the swap space part of "killed: out of swap space"
> > can be a misnomer. Killing is driven by having low free RAM
> > for sufficiently long. vm.pageout_oom_seq controls how
> > long. Swap space may be unused, little used, or actually be
> > low. With 3 GiBytes of swap space in the partition, these
> > runs were not low on swap space.)
> >
> > Even with fairly stable ms/w figures the queue depths can
> > get to be large, implying long times before the last queued
> > write completes. For example (for a USB storage device for
> > the root file system and swap parti

Re: building LLVM threads gets killed

[blubee blubeeme]

On Mon, Aug 20, 2018 at 11:04 PM Mark Millard  wrote:

> Rodney W. Grimes freebsd-rwg at pdx.rh.CN85.dnsmgr.net wrote on
> Mon Aug 20 14:26:55 UTC 2018 :
>
> > > On 20 Aug 2018, at 05:01, blubee blubeeme 
> wrote:
> > > >
> > > > I am running current compiling LLVM60 and when it comes to linking
> > > > basically all the processes on my computer gets killed; Chrome,
> Firefox and
> > > > some of the LLVM threads as well
> > > ...
> > > > llvm/build % ninja -j8
> > > > [2408/2473] Building CXX object
> > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > > FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > > /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> > > > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes
> -I../lib/Passes
> > > > -Iinclude -I../include -isystem /usr/local/include -fPIC
> > > > -fvisibility-inlines-hidden -Werror=date-time
> > > > -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> > > > -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> > > > -Wmissing-field-initializers -pedantic -Wno-long-long
> > > > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> > > > -Wstring-conversion -fdiagnostics-color -g-fno-exceptions
> -fno-rtti -MD
> > > > -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> > > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> > > > ../lib/Passes/PassBuilder.cpp
> > > > c++: error: unable to execute command: Killed
> > >
> > > It is running out of RAM while running multiple parallel link jobs.  If
> > > you are building using WITH_DEBUG, turn that off, it consumes large
> > > amounts of memory.  If you must have debug info, try adding the
> > > following flag to the CMake command line:
> > >
> > > -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> > >
> > > That will limit the amount of parallel link jobs to 1, even if you
> > > specify -j 8 to gmake or ninja.
> > >
> > > Brooks, it would not be a bad idea to always use this CMake flag in the
> > > llvm ports. :)
> >
> > And this may also fix the issues that all the small
> > memory (aka, RPI*) buliders are facing when trying
> > to do -j4?
>
> It may help, but:
>
> Even just compiles with no links running can get the kills
> in such small system contexts.
>
> And going for a simpler context that can demonstrate
> the behavior . . .
>
> Taking a Pine64+ 2GB as an example (4 cores with 1
> HW-thread per core, 2 GiBytes of RAM, USB device for
> root file system and swap partition):
>
> In another login:
> # stress -d 2 -m 4 --vm-keep --vm-bytes 536870912
>
> That "4" and "536870912" total to the 2 GiBytes so
> swapping is induced for the context in question.
> (Scale --vm-bytes appropriately to context.)
> [Note: I had 3 GiBytes of swap space in a partition
> for the below.]
>
> [stress is from the port sysutils/stress .]
>
> I had left the default vm.pageout_oom_seq=12 in place for this,
> making the kills easier to get than the 120 figure would. It
> does not take very long generally for some sort of message to
> show up. Sometimes kills happen:
>
> My test environment has Mark Johnston patches to report
> things not normally reported:
>
> waited 9s for async swap write
> waited 9s for swap buffer
> waited 9s for async swap write
> waited 9s for async swap write
> waited 9s for async swap write
> v_free_count: 1357, v_inactive_count: 1
> Aug 20 06:04:27 pine64 kernel: pid 1010 (stress), uid 0, was killed: out
> of swap space
> waited 5s for async swap write
> waited 5s for swap buffer
> waited 5s for async swap write
> waited 5s for async swap write
> waited 5s for async swap write
> waited 13s for async swap write
> waited 12s for swap buffer
> waited 13s for async swap write
> waited 12s for async swap write
> waited 12s for async swap write
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 161177, size: 131072
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164766, size: 65536
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164064, size: 12288
> . . .
>
> (I made multiple runs, most manually stopped. None run for
> all that long.)
>
> (Warning: the swap space part of "killed: out of swap space"
> can be a misnomer. Killing is driven by having low free RAM
> for sufficiently long. vm.pageout_oom_seq controls how
> long. Swap space may be unused, little used, or actually be
> low. With 3 GiBytes of swap space in the partition, these
> runs were not low on swap space.)
>
> Even with fairly stable ms/w figures the queue depths can
> get to be large, implying long times before the last queued
> write completes. For example (for a USB storage device for
> the root file system and swap partition):
>
> dT: 1.006s  w: 1.000s
> L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/wd/s   kBps
>  ms/d   %busy Name
>0  0  0  00.0  0  00.0  0  0
> 0.00.0| mmcsd0
>   56312  0   

soundcard.h expose device block size

[blubee blubeeme]

I'm looking at FreeBSD's soundcard.h and it doesn't seem to expose the
hardware block size, this seems like a limitation stemming from FreeBSD's
soundcard.h being an extension of OSS.

Since we've already implemented a lot of additional features on top of the
OSS API, is it possible to also expose the hardware block size for
applications that might need access to that information?

Best,
Owen
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Newly upgraded -CURRENT box does not boot

[Warner Losh]

On Mon, Aug 20, 2018 at 5:24 PM, Brett 
wrote:

> On 8/20/2018 7:09 PM, Warner Losh wrote:
>
>> On Mon, Aug 20, 2018 at 4:50 PM, Brett 
>> wrote:
>>
>> Hi Kevin,
>>>
>>> Thanks for your help.
>>>
>>> Should I be doing "make LOADER_DEFAULT_INTERP=4th buildkernel && make
>>> LOADER_DEFAULT_INTERP=4th installkernel"?
>>>
>>> Nope. Kernel has nothing to do with it. That won't work.
>>
>> I had also previously tried " make clean all install
>>
>>> WITHOUT_LUA_LOADER=yes" in /usr/src/stand which did not help.
>>>
>>> You could just do "ln -sf /boot/loader_4th /boot/loader" and reboot if
>> that's the issue. Or at the boot2 prompt, you could type in
>> /boot/loader_4th if you haven't booted yet.
>>
>> You could also do a simple 'make install LOADER_DEFAULT_INTERP=4th" which
>> does approximately the same thing.
>>
>> Though why you'd get loader.lua not found is kinda crazy
>>
>> Warner
>>
> Hi Warner,
>
> I've tried those things, unfortunately none of them worked. With forth,
> things are slightly different (no more lua error), but ultimately the same
> result: - the kernel doesn't load ("can't load 'kernel'").
>
> I lastly tried "ln -sf /boot/loader_4th /boot/loader", now I get "No
> /boot/loader" and a boot: prompt. I can do "boot/loader_4th" which will use
> the forth loader with the results the same as the above.
>
> Any help is appreciated - this is a very important box unfortunately. :(
>

If both /boot/loader_lua and /boot/loader_4th fail in the same way, you may
be in trouble. There's a sliver of a chance that /boot/loader_simp might
work, but if the other two both fail, it's something common between them
that's likely shared by /boot/loader_simp.

I'd install old /boot/loader and then try to bisect where things went wrong
from your last update. Thankfully, it looks like the last known good one
for you was in March, and all versions since then you can rebuild just
src/stand w/o doing a full world so it should be quick to trace down. There
are maybe 20,000 svn changes or so, which should zero in on the right one
in about 17 tries... It takes me about a minute to build, so with luck
you'll know in about an hour...

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Newly upgraded -CURRENT box does not boot

[Brett]

On 8/20/2018 7:09 PM, Warner Losh wrote:

On Mon, Aug 20, 2018 at 4:50 PM, Brett 
wrote:


Hi Kevin,

Thanks for your help.

Should I be doing "make LOADER_DEFAULT_INTERP=4th buildkernel && make
LOADER_DEFAULT_INTERP=4th installkernel"?


Nope. Kernel has nothing to do with it. That won't work.

I had also previously tried " make clean all install

WITHOUT_LUA_LOADER=yes" in /usr/src/stand which did not help.


You could just do "ln -sf /boot/loader_4th /boot/loader" and reboot if
that's the issue. Or at the boot2 prompt, you could type in
/boot/loader_4th if you haven't booted yet.

You could also do a simple 'make install LOADER_DEFAULT_INTERP=4th" which
does approximately the same thing.

Though why you'd get loader.lua not found is kinda crazy

Warner

Hi Warner,

I've tried those things, unfortunately none of them worked. With forth, 
things are slightly different (no more lua error), but ultimately the 
same result: - the kernel doesn't load ("can't load 'kernel'").


I lastly tried "ln -sf /boot/loader_4th /boot/loader", now I get "No 
/boot/loader" and a boot: prompt. I can do "boot/loader_4th" which will 
use the forth loader with the results the same as the above.


Any help is appreciated - this is a very important box unfortunately. :(

Thanks,

-Brett

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Newly upgraded -CURRENT box does not boot

[Warner Losh]

On Mon, Aug 20, 2018 at 4:50 PM, Brett 
wrote:

> Hi Kevin,
>
> Thanks for your help.
>
> Should I be doing "make LOADER_DEFAULT_INTERP=4th buildkernel && make
> LOADER_DEFAULT_INTERP=4th installkernel"?
>

Nope. Kernel has nothing to do with it. That won't work.

I had also previously tried " make clean all install
> WITHOUT_LUA_LOADER=yes" in /usr/src/stand which did not help.
>

You could just do "ln -sf /boot/loader_4th /boot/loader" and reboot if
that's the issue. Or at the boot2 prompt, you could type in
/boot/loader_4th if you haven't booted yet.

You could also do a simple 'make install LOADER_DEFAULT_INTERP=4th" which
does approximately the same thing.

Though why you'd get loader.lua not found is kinda crazy

Warner


> Thanks!
>
> -Brett
>
> On 8/20/2018 6:31 PM, Kevin Oberman wrote:
>
>> Check the archive of this list for the past 24 hours. The default bot was
>> just converted to the lua boot today and there have been some issues. The
>> "one liner" to switch back to the FORTH boot was posted to use to work
>> around this and I believe that the change will be rolled back until the
>> "corner cases" can be fixed.
>> http://lists.freebsd.org/pipermail/freebsd-current/
>> --
>> Kevin Oberman, Part time kid herder and retired Network Engineer
>> E-mail: rkober...@gmail.com 
>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
>>
>>
>>
>> On Mon, Aug 20, 2018 at 2:28 PM Brett Gmoser <
>> freebsdcurr...@codexterous.com >
>> wrote:
>>
>> Hi there,
>>
>> I was told to e-mail these addresses with this.
>>
>> I did an `svn update` on /usr/src last night, build world and
>> kernel as
>> usual. This morning I installed the kernel, booted into single user,
>> installed world and did mergemaster -Ui as usual. The new kernel had
>> booted fine. Upon reboot, the machine will no longer boot:
>>
>>  Startup error in /boot/lua/loader.lua:
>>  LUA ERROR: cannot open /boot/lua/loader.lua: no such file or
>> directory
>>
>>  can't load 'kernel'
>>
>> Many things in the bootloader do not work, including "boot
>> kernel.old",
>> "ls /boot", and various other things (most if not all just result in
>> "Command failed"). Interestingly, "ls /mnt" works, other
>> directories do
>> not. That's the only clue I have.
>>
>> I'm able to reboot in an installer image and mount the drive just
>> fine.
>> Everything is there and is as expected, including
>> /boot/lua/loader.lua.
>>
>> I re-installed everything in /usr/src/stand (chroot'd on the
>> installer
>> image, and "cd /usr/src/stand && make clean all install"). This
>> did not
>> fix the problem.
>>
>> Does anybody happen to have any ideas?
>>
>> Thanks,
>>
>> -Brett
>>
>> ___
>> freebsd-current@freebsd.org 
>> mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to
>> "freebsd-current-unsubscr...@freebsd.org
>> "
>>
>>
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Newly upgraded -CURRENT box does not boot

[Brett]

Hi Kevin,

Thanks for your help.

Should I be doing "make LOADER_DEFAULT_INTERP=4th buildkernel && make 
LOADER_DEFAULT_INTERP=4th installkernel"?


I had also previously tried " make clean all install 
WITHOUT_LUA_LOADER=yes" in /usr/src/stand which did not help.


Thanks!

-Brett

On 8/20/2018 6:31 PM, Kevin Oberman wrote:
Check the archive of this list for the past 24 hours. The default bot 
was just converted to the lua boot today and there have been some 
issues. The "one liner" to switch back to the FORTH boot was posted to 
use to work around this and I believe that the change will be rolled 
back until the "corner cases" can be fixed.

http://lists.freebsd.org/pipermail/freebsd-current/
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com 
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683


On Mon, Aug 20, 2018 at 2:28 PM Brett Gmoser 
> wrote:


Hi there,

I was told to e-mail these addresses with this.

I did an `svn update` on /usr/src last night, build world and
kernel as
usual. This morning I installed the kernel, booted into single user,
installed world and did mergemaster -Ui as usual. The new kernel had
booted fine. Upon reboot, the machine will no longer boot:

 Startup error in /boot/lua/loader.lua:
 LUA ERROR: cannot open /boot/lua/loader.lua: no such file or
directory

 can't load 'kernel'

Many things in the bootloader do not work, including "boot
kernel.old",
"ls /boot", and various other things (most if not all just result in
"Command failed"). Interestingly, "ls /mnt" works, other
directories do
not. That's the only clue I have.

I'm able to reboot in an installer image and mount the drive just
fine.
Everything is there and is as expected, including
/boot/lua/loader.lua.

I re-installed everything in /usr/src/stand (chroot'd on the
installer
image, and "cd /usr/src/stand && make clean all install"). This
did not
fix the problem.

Does anybody happen to have any ideas?

Thanks,

-Brett

___
freebsd-current@freebsd.org 
mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org
"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Fatal trap 12: page fault on Acer Chromebook 720 (peppy)

[Michael Gmelin]

> On 20. Aug 2018, at 17:09, Konstantin Belousov  wrote:
> 
>> On Mon, Aug 20, 2018 at 12:45:12AM +0200, Michael Gmelin wrote:
>> 
>> See here for a screenshot (also including the output of "show pte
>> 0xf8000100"):
>> 
>> https://gist.github.com/grembo/78d0f2a100dd4f16775b85a118769658#file-ddb1-png
> It is too early for ddb routines to register.
> Ok can you try the following debugging patch, to verify my guess ?
> 
> diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c
> index 18777d23f09..cd05fdb763f 100644
> --- a/sys/amd64/amd64/pmap.c
> +++ b/sys/amd64/amd64/pmap.c
> @@ -1052,8 +1052,7 @@ create_pagetables(vm_paddr_t *firstaddr)
>pd_p = (pd_entry_t *)DMPDkernphys;
>for (i = 0; i < (NPDEPG * nkdmpde); i++)
>pd_p[i] = (i << PDRSHIFT) | X86_PG_V | PG_PS | pg_g |
> -X86_PG_M | X86_PG_A | pg_nx |
> -bootaddr_rwx(i << PDRSHIFT);
> +X86_PG_M | X86_PG_A | pg_nx | X86_PG_RW;
>for (i = 0; i < nkdmpde; i++)
>pdp_p[i] = (DMPDkernphys + ptoa(i)) | X86_PG_RW |
>X86_PG_V;

With this change it boots okay (mptramp_pagetables is 0x100, as expected).

Best,
Michael


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Sharing compiled builds between multiple 12-CURRENT boxes.

[Dhananjay Balan]

On Mon, Aug 20, 2018 at 05:48:04AM -0700, Cy Schubert wrote:

> Make sure the archs are the same. Make sure the src and obj pathnames 
> on both servers are exactly the same. Make sure /usr/src (or wherever 
> you put it) is the same, it too needs to be rsynced.
> 
> The tiniest differences will lead to this error.

Starting from a make clean and rsyncing fixed the problems, I can
install and boot my laptop from the build on my server.

Thanks!
Dhananjay Balan
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

[O. Hartmann]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Am Mon, 20 Aug 2018 21:24:21 +0200
"O. Hartmann"  schrieb:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> Building NanoBSD world on CURRENT r338113 fails due to:
> 
> [...]
> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&  MK_TESTS=no
> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE 
> CRUNCH_CFLAGS=-DRESCUE
> MK_AUTO_OBJ=no   obj make[5]: 
> "/pool/sources/CURRENT/src/tools/build/mk/Makefile.boot"
> line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.  Copy 
> the header to
> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused by 
> Makefile
> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error code 1
> 
> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
> [...]
> 
> 
> This problem occured during today's source updates since I was able to build 
> the NanoBSD
> image I intend to build yesterday ~ r338060.
> 
> What is going wrong?

It seems the problem has been introduced after r338095, since r338095 builds 
ok, while
r338096 doesn't.

> 
> I've set the following for NanoBSD's build and install:
> 
> # BUILDWORLD ONLY
> # Options to put in make.conf during buildworld only
> CONF_BUILD='
> CFLAGS=-O3 -pipe
> MALLOC_PRODUCTION=YES
> INSTALL_NODEBUG=YES
> '
> # Options to put in make.conf during installworld only
> CONF_INSTALL='
> WITHOUT_TOOLCHAIN=YES
> WITHOUT_CROSS_COMPILER=YES
> WITHOUT_DOCCOMPRESS=YES
> WITHOUT_INSTALLLIB=YES
> WITHOUT_BINUTILS=YES
> #
> #WITHOUT_ACCT=YES
> WITHOUT_AMD=YES
> WITHOUT_APM=YES
> WITHOUT_ASSERT_DEBUG=YES
> WITHOUT_AT=YES
> WITHOUT_ATM=YES
> WITHOUT_BHYVE=YES
> WITHOUT_BLUETOOTH=YES
> WITHOUT_BOOTPARAMD=YES
> WITHOUT_BOOTPD=YES
> WITHOUT_BSDINSTALL=YES
> #WITHOUT_BSD_CPIO=YES
> #WITHOUT_BSNMP=YES
> #WITHOUT_BZIP2=YES
> WITHOUT_CALENDAR=YES
> #WITHOUT_CCD=YES
> WITHOUT_CDDL=YES
> WITHOUT_CLANG_FULL=YES
> WITHOUT_CTM=YES
> #WITHOUT_CUSE=YES
> WITHOUT_CXX=YES
> WITHOUT_DICT=YES
> WITHOUT_DIALOG=YES
> WITHOUT_EXAMPLES=YES
> WITHOUT_EE=YES
> WITHOUT_FINGER=YES
> WITHOUT_FLOPPY=YES
> WITHOUT_FREEBSD_UPDATE=YES
> WITHOUT_FDT=YES
> WITHOUT_GAMES=YES
> WITHOUT_GCOV=YES
> WITHOUT_GDB=YES
> WITHOUT_HAST=YES
> WITHOUT_HTML=YES
> WITHOUT_HYPERV=YES
> #WITHOUT_INET6=YES
> #WITHOUT_INETD=YES
> WITHOUT_IPFILTER=YES
> WITHOUT_ISCSI=YES
> WITHOUT_KDUMP=YES
> WITHOUT_KERNEL_SYMBOLS=YES
> #WITHOUT_LDNS=YES
> WITHOUT_LOCATE=YES
> WITHOUT_LPR=YES
> #WITHOUT_MAIL=YES
> #WITHOUT_MAILWRAPPER=YES
> #WITHOUT_MAKE=YES
> WITHOUT_MAN=YES
> WITHOUT_MAN_UTILS=YES
> WITHOUT_NDIS=YES
> #WITHOUT_NETCAT=YES
> #WITHOUT_NIS=YES
> #WITHOUT_NLS_CATALOGS=YES
> #WITHOUT_NS_CACHING=YES
> WITHOUT_NAND=YES
> WITHOUT_OFED=YES
> WITHOUT_PC_SYSINSTALL=YES
> WITHOUT_PF=YES
> WITHOUT_PKGBOOTSTRAP=YES
> WITHOUT_PMC=YES
> WITHOUT_PORTSNAP=YES
> #WITHOUT_PPP=YES
> WITHOUT_QUOTAS=YES
> WITHOUT_RBOOTD=YES
> WITHOUT_RCMDS=YES
> #WITHOUT_RESCUE=YES
> #WITHOUT_ROUTED=YES
> #WITHOUT_SENDMAIL=YES
> WITHOUT_SHAREDOCS=YES
> WITHOUT_SVNLITE=YES
> WITHOUT_TALK=YES
> #WITHOUT_TCSH=YES
> WITHOUT_TELNET=YES
> WITHOUT_TEXTPROC=YES
> WITHOUT_TFTP=YES
> #WITHOUT_WIRELESS=YES
> WITHOUT_ZFS=YES
> '
> 
> # BUILD and INSTALL!
> # Options to put in make.conf during both build- & installworld.
> # See man src.conf(5) for more details on WITHOUT_ tags.
> CONF_WORLD='
> WITH_KERNEL_RETPOLINE=YES
> WITH_LLVM_TARGET_BPF=YES
> WITHOUT_TESTS=YES
> WITHOUT_TESTS_SUPPORT=YES
> WITHOUT_ASSERT_DEBUG=YES
> WITHOUT_CCD=YES
> #WITHOUT_BINUTILS_BOOTSTRAP=YES
> WITHOUT_DEBUG_FILES=YES
> #WITHOUT_ICONV=YES
> WITHOUT_ISCSI=YES
> WITHOUT_LIB32=YES
> #WITHOUT_LLVM_TARGET_ALL=YES
> WITH_ZONEINFO_LEAPSECONDS_SUPPORT=YES
> WITHOUT_SVN=YES
> WITH_SORT_THREADS=YES
> WITH_EXTRA_TCP_STACKS=YES
> #
> INSTALL_NODEBUG=YES
> '
> 
> 
> 
> - -- 
> O. Hartmann
> 
> Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
> Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).
> -BEGIN PGP SIGNATURE-
> 
> iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCW3sVgAAKCRDS528fyFhY
> lL4MAgCaS8UeE6FwJIeUa5Q+9hf80DJkde+oopttIL2Jz9LcbN1CRU5JTc2jJ9Vb
> vO380RVTNO4M0Ge0M0m2wDCx4xzuAf9YYdjW/xTXbh09YcTlt3l22pn1aaHYrdMk
> pOYA3FUw8xi09cniyQHBOr4mvjKI07GtgLKTEINU3SuS9CmFjCzg
> =vFP+
> -END PGP SIGNATURE-
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



- -- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).
-BEGIN PGP SIGNATURE-

iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCW3sd/wAKCRDS528fyFhY
lM7PAf4oZ05VCbM8DZXqUz23f/eYlU57DOTJRmHrC7PUisD8GR4YananlyvtmCz/
H40KXUxsg2820cVv5n6

Newly upgraded -CURRENT box does not boot

[Brett Gmoser]

Hi there,

I was told to e-mail these addresses with this.

I did an `svn update` on /usr/src last night, build world and kernel as 
usual. This morning I installed the kernel, booted into single user, 
installed world and did mergemaster -Ui as usual. The new kernel had 
booted fine. Upon reboot, the machine will no longer boot:


    Startup error in /boot/lua/loader.lua:
    LUA ERROR: cannot open /boot/lua/loader.lua: no such file or directory

    can't load 'kernel'

Many things in the bootloader do not work, including "boot kernel.old", 
"ls /boot", and various other things (most if not all just result in 
"Command failed"). Interestingly, "ls /mnt" works, other directories do 
not. That's the only clue I have.


I'm able to reboot in an installer image and mount the drive just fine. 
Everything is there and is as expected, including /boot/lua/loader.lua.


I re-installed everything in /usr/src/stand (chroot'd on the installer 
image, and "cd /usr/src/stand && make clean all install"). This did not 
fix the problem.


Does anybody happen to have any ideas?

Thanks,

-Brett

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: CFT: TRIM Consolodation on UFS/FFS filesystems

[Kirk McKusick]

From: Kirk McKusick 
To: FreeBSD Current ,
FreeBSD Filesystems 
Subject: CFT: TRIM Consolodation on UFS/FFS filesystems
Date: Mon, 20 Aug 2018 12:40:56 -0700

Oops, forgot that attachments get stripped. Below are the diffs for
gathering statistics. Sorry to those of you on Gmail for whom they
will be mangled.

Kirk McKusick

=-=-=

Index: sbin/mount/mount.c
===
--- sbin/mount/mount.c  (revision 338054)
+++ sbin/mount/mount.c  (working copy)
@@ -686,6 +686,18 @@ prmount(struct statfs *sfp)
for (i = 0; i < sizeof(sfp->f_fsid); i++)
printf("%02x", ((u_char *)&sfp->f_fsid)[i]);
}
+   if (sfp->f_trim_total != 0 || sfp->f_trim_total_blks != 0)
+   (void)printf(", TRIM: total %ju total blocks %ju",
+   (uintmax_t)sfp->f_trim_total,
+   (uintmax_t)sfp->f_trim_total_blks);
+   if (sfp->f_trim_inflight != 0 || sfp->f_trim_inflight_blks != 0)
+   (void)printf(", TRIM: inflight %ju inflight blocks %ju",
+   (uintmax_t)sfp->f_trim_inflight,
+   (uintmax_t)sfp->f_trim_inflight_blks);
+   if (sfp->f_pendingblks != 0 || sfp->f_pendingfiles != 0)
+   (void)printf(", pending blocks %ju, pending files %ju",
+   (uintmax_t)sfp->f_pendingblks,
+   (uintmax_t)sfp->f_pendingfiles);
}
(void)printf(")\n");
 }
Index: sys/sys/mount.h
===
--- sys/sys/mount.h (revision 338054)
+++ sys/sys/mount.h (working copy)
@@ -85,7 +85,13 @@ struct statfs {
uint64_t f_asyncwrites; /* count of async writes since mount */
uint64_t f_syncreads;   /* count of sync reads since mount */
uint64_t f_asyncreads;  /* count of async reads since mount */
-   uint64_t f_spare[10];   /* unused spare */
+   uint64_t f_trim_total;  /* count of TRIM ops since mount */
+   uint64_t f_trim_total_blks; /* count of TRIM blocks since mount */
+   uint64_t f_trim_inflight;   /* count of TRIM ops in progress */
+   uint64_t f_trim_inflight_blks;  /* count of TRIM blocks in progress */
+   int64_t  f_pendingblks; /* pending free blocks */
+   int64_t  f_pendingfiles;/* pending free nodes */
+   uint64_t f_spare[4];/* unused spare */
uint32_t f_namemax; /* maximum filename length */
uid_t f_owner;  /* user that mounted the filesystem */
fsid_tf_fsid;   /* filesystem id */
Index: sys/ufs/ffs/ffs_vfsops.c
===
--- sys/ufs/ffs/ffs_vfsops.c(revision 338081)
+++ sys/ufs/ffs/ffs_vfsops.c(working copy)
@@ -1398,7 +1398,13 @@ ffs_statfs(mp, sbp)
sbp->f_bsize = fs->fs_fsize;
sbp->f_iosize = fs->fs_bsize;
sbp->f_blocks = fs->fs_dsize;
+   sbp->f_pendingblks = dbtofsb(fs, fs->fs_pendingblocks);
+   sbp->f_pendingfiles = fs->fs_pendinginodes;
UFS_LOCK(ump);
+   sbp->f_trim_total = ump->um_trim_total;
+   sbp->f_trim_total_blks = ump->um_trim_total_blks;
+   sbp->f_trim_inflight = ump->um_trim_inflight;
+   sbp->f_trim_inflight_blks = ump->um_trim_inflight_blks;
sbp->f_bfree = fs->fs_cstotal.cs_nbfree * fs->fs_frag +
fs->fs_cstotal.cs_nffree + dbtofsb(fs, fs->fs_pendingblocks);
sbp->f_bavail = freespace(fs, fs->fs_minfree) +

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

CFT: TRIM Consolodation on UFS/FFS filesystems

[Kirk McKusick]

I have recently added TRIM consolodation support for the UFS/FFS
filesystem. This feature consolodates large numbers of TRIM commands
into a much smaller number of commands covering larger blocks of
disk space. Best described by the commit message:

  Author: mckusick
  Date: Sun Aug 19 16:56:42 2018
  New Revision: 338056
  URL: https://svnweb.freebsd.org/changeset/base/338056

  Log:
Add consolodation of TRIM / BIO_DELETE commands to the UFS/FFS filesystem.

When deleting files on filesystems that are stored on flash-memory
(solid-state) disk drives, the filesystem notifies the underlying
disk of the blocks that it is no longer using. The notification
allows the drive to avoid saving these blocks when it needs to
flash (zero out) one of its flash pages. These notifications of
no-longer-being-used blocks are referred to as TRIM notifications.
In FreeBSD these TRIM notifications are sent from the filesystem
to the drive using the BIO_DELETE command.

Until now, the filesystem would send a separate message to the drive
for each block of the file that was deleted. Each Gigabyte of file
size resulted in over 3000 TRIM messages being sent to the drive.
This burst of messages can overwhelm the drive's task queue causing
multiple second delays for read and write requests.

This implementation collects runs of contiguous blocks in the file
and then consolodates them into a single BIO_DELETE command to the
drive. The BIO_DELETE command describes the run of blocks as a
single large block being deleted. Each Gigabyte of file size can
result in as few as two BIO_DELETE commands and is typically less
than ten.  Though these larger BIO_DELETE commands take longer to
run, they do not clog the drive task queue, so read and write
commands can intersperse effectively with them.

Though this new feature has been throughly reviewed and tested, it
is being added disabled by default so as to minimize the possibility
of disrupting the upcoming 12.0 release. It can be enabled by running
``sysctl vfs.ffs.dotrimcons=1''. Users are encouraged to test it.
If no problems arise, we will consider requesting that it be enabled
by default for 12.0.

Reviewed by:  kib
Tested by:Peter Holm
Sponsored by: Netflix

This support is off by default, but I am hoping that I can get enough
testing to ensure that it (a) works, and (b) is helpful that it will
be reasonable to have it turned on by default in 12.0. The cutoff for
turning it on by default in 12.0 is September 19th. So I am requesting
your testing feedback in the near-term. Please let me know if you have
managed to use it successfully (or not) and also if it provided any
performance difference (good or bad).

To enable TRIM consolodation either use `sysctl vfs.ffs.dotrimcons=1'
or just set the `dotrimcons' variable in sys/ufs/ffs/ffs_alloc.c to 1.

Everything you need to test TRIM consolodation is obtained by setting
the above sysctl. However, if you want to collect statistics on how
effective the TRIM consolodation is working, the attached diff will
allow you to easily get statitics on how the TRIM is going. Compile your
kernel and the mount command. Note that if you do not do a buildworld,
you will need to copy /sys/sys/mount.h to /usr/include/sys/mount.h to
get the patched mount command to compile. Then run `mount -v'
(or `mount -v | grep /mnt' to get just the statistics for /mnt).

Removing a 30Mb file without TRIM consolodation:
/dev/md0 on /mnt (ufs, local, writes: sync 10 async 482, reads: sync 7 async 0, 
fsid d43f795b6a7d34fb, TRIM: total 952 total blocks 7616)

While removing the same file with TRIM consolodation:
/dev/md0 on /mnt (ufs, local, writes: sync 10 async 482, reads: sync 7 async 0, 
fsid d43f795b6a7d34fb, TRIM: total 3 total blocks 7616)

It also tracks pending blocks and pending files. These numbers are only
printed out when they are non-zero. Here is an example running with soft
updates right after a file has been rm'ed, but its blocks not yet released:
/dev/md0 on /mnt (ufs, local, soft-updates, writes: sync 2 async 251, reads: 
sync 5 async 0, fsid 303f795b1be0c459, pending blocks 7616, pending files 1)

Finally it tracks inflight BIO_DELETEs and total blocks represented by
those inflight BIO_DELETEs. These numbers are also only printed out when
they are non-zero. These statistics let you see how much of a backlog
of BIO_DELETEs you have backed up at/in the disk drive and you can track
how quickly they drain.

Kirk McKusick

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Loader post r338064 deletes GOP screen when asking geli passphrase

[Kyle Evans]

On Mon, Aug 20, 2018 at 1:01 PM, Harry Schmalzbauer  wrote:
> Hello,
>
> as far as I understand, r338064 should bring back default loader
> environment.
>
> I'm UEFI booting r338093/amd64 (LynxPoint/Haswell).
>
> Since today, the GOP-console output of the first loader lines get deleted
> when the geli passphrase prompt shows up.
>
> Nothing uncommon here, and never seen that before, but I guess this can be
> considered a bug.
> Or is that an intended behaviour?
>

It is intended behavior, but I won't claim that it isn't a bug. =) I
*think* I added the screen clear prior to going into the passphrase
prompt because the cursor position was ending up all wrong and it the
passphrase prompt was obscured by lots of noisy text that it ended up
drawing in the middle of.

I'm certainly willing to go back on that position if there turns out
to be a better solution, one of which I think I see.

Thanks,

Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

[O. Hartmann]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Building NanoBSD world on CURRENT r338113 fails due to:

[...]
cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&  MK_TESTS=no
UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE 
CRUNCH_CFLAGS=-DRESCUE
MK_AUTO_OBJ=no   obj make[5]: 
"/pool/sources/CURRENT/src/tools/build/mk/Makefile.boot"
line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.  Copy the 
header to
${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused by 
Makefile
in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error code 1

make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
[...]


This problem occured during today's source updates since I was able to build 
the NanoBSD
image I intend to build yesterday ~ r338060.

What is going wrong?

I've set the following for NanoBSD's build and install:

# BUILDWORLD ONLY
# Options to put in make.conf during buildworld only
CONF_BUILD='
CFLAGS=-O3 -pipe
MALLOC_PRODUCTION=YES
INSTALL_NODEBUG=YES
'
# Options to put in make.conf during installworld only
CONF_INSTALL='
WITHOUT_TOOLCHAIN=YES
WITHOUT_CROSS_COMPILER=YES
WITHOUT_DOCCOMPRESS=YES
WITHOUT_INSTALLLIB=YES
WITHOUT_BINUTILS=YES
#
#WITHOUT_ACCT=YES
WITHOUT_AMD=YES
WITHOUT_APM=YES
WITHOUT_ASSERT_DEBUG=YES
WITHOUT_AT=YES
WITHOUT_ATM=YES
WITHOUT_BHYVE=YES
WITHOUT_BLUETOOTH=YES
WITHOUT_BOOTPARAMD=YES
WITHOUT_BOOTPD=YES
WITHOUT_BSDINSTALL=YES
#WITHOUT_BSD_CPIO=YES
#WITHOUT_BSNMP=YES
#WITHOUT_BZIP2=YES
WITHOUT_CALENDAR=YES
#WITHOUT_CCD=YES
WITHOUT_CDDL=YES
WITHOUT_CLANG_FULL=YES
WITHOUT_CTM=YES
#WITHOUT_CUSE=YES
WITHOUT_CXX=YES
WITHOUT_DICT=YES
WITHOUT_DIALOG=YES
WITHOUT_EXAMPLES=YES
WITHOUT_EE=YES
WITHOUT_FINGER=YES
WITHOUT_FLOPPY=YES
WITHOUT_FREEBSD_UPDATE=YES
WITHOUT_FDT=YES
WITHOUT_GAMES=YES
WITHOUT_GCOV=YES
WITHOUT_GDB=YES
WITHOUT_HAST=YES
WITHOUT_HTML=YES
WITHOUT_HYPERV=YES
#WITHOUT_INET6=YES
#WITHOUT_INETD=YES
WITHOUT_IPFILTER=YES
WITHOUT_ISCSI=YES
WITHOUT_KDUMP=YES
WITHOUT_KERNEL_SYMBOLS=YES
#WITHOUT_LDNS=YES
WITHOUT_LOCATE=YES
WITHOUT_LPR=YES
#WITHOUT_MAIL=YES
#WITHOUT_MAILWRAPPER=YES
#WITHOUT_MAKE=YES
WITHOUT_MAN=YES
WITHOUT_MAN_UTILS=YES
WITHOUT_NDIS=YES
#WITHOUT_NETCAT=YES
#WITHOUT_NIS=YES
#WITHOUT_NLS_CATALOGS=YES
#WITHOUT_NS_CACHING=YES
WITHOUT_NAND=YES
WITHOUT_OFED=YES
WITHOUT_PC_SYSINSTALL=YES
WITHOUT_PF=YES
WITHOUT_PKGBOOTSTRAP=YES
WITHOUT_PMC=YES
WITHOUT_PORTSNAP=YES
#WITHOUT_PPP=YES
WITHOUT_QUOTAS=YES
WITHOUT_RBOOTD=YES
WITHOUT_RCMDS=YES
#WITHOUT_RESCUE=YES
#WITHOUT_ROUTED=YES
#WITHOUT_SENDMAIL=YES
WITHOUT_SHAREDOCS=YES
WITHOUT_SVNLITE=YES
WITHOUT_TALK=YES
#WITHOUT_TCSH=YES
WITHOUT_TELNET=YES
WITHOUT_TEXTPROC=YES
WITHOUT_TFTP=YES
#WITHOUT_WIRELESS=YES
WITHOUT_ZFS=YES
'

# BUILD and INSTALL!
# Options to put in make.conf during both build- & installworld.
# See man src.conf(5) for more details on WITHOUT_ tags.
CONF_WORLD='
WITH_KERNEL_RETPOLINE=YES
WITH_LLVM_TARGET_BPF=YES
WITHOUT_TESTS=YES
WITHOUT_TESTS_SUPPORT=YES
WITHOUT_ASSERT_DEBUG=YES
WITHOUT_CCD=YES
#WITHOUT_BINUTILS_BOOTSTRAP=YES
WITHOUT_DEBUG_FILES=YES
#WITHOUT_ICONV=YES
WITHOUT_ISCSI=YES
WITHOUT_LIB32=YES
#WITHOUT_LLVM_TARGET_ALL=YES
WITH_ZONEINFO_LEAPSECONDS_SUPPORT=YES
WITHOUT_SVN=YES
WITH_SORT_THREADS=YES
WITH_EXTRA_TCP_STACKS=YES
#
INSTALL_NODEBUG=YES
'



- -- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).
-BEGIN PGP SIGNATURE-

iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCW3sVgAAKCRDS528fyFhY
lL4MAgCaS8UeE6FwJIeUa5Q+9hf80DJkde+oopttIL2Jz9LcbN1CRU5JTc2jJ9Vb
vO380RVTNO4M0Ge0M0m2wDCx4xzuAf9YYdjW/xTXbh09YcTlt3l22pn1aaHYrdMk
pOYA3FUw8xi09cniyQHBOr4mvjKI07GtgLKTEINU3SuS9CmFjCzg
=vFP+
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Loader post r338064 deletes GOP screen when asking geli passphrase

[Harry Schmalzbauer]

Hello,

as far as I understand, r338064 should bring back default loader 
environment.


I'm UEFI booting r338093/amd64 (LynxPoint/Haswell).

Since today, the GOP-console output of the first loader lines get 
deleted when the geli passphrase prompt shows up.


Nothing uncommon here, and never seen that before, but I guess this can 
be considered a bug.

Or is that an intended behaviour?

Thanks,

-harry

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[bob prohaska]

On Mon, Aug 20, 2018 at 07:33:32PM +0200, Dimitry Andric wrote:
> On 20 Aug 2018, at 16:26, Rodney W. Grimes 
>  wrote:
> > 
> >> It is running out of RAM while running multiple parallel link jobs.  If
> >> you are building using WITH_DEBUG, turn that off, it consumes large
> >> amounts of memory.  If you must have debug info, try adding the
> >> following flag to the CMake command line:
> >> 
> >> -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> >> 
> >> That will limit the amount of parallel link jobs to 1, even if you
> >> specify -j 8 to gmake or ninja.
> >> 
> >> Brooks, it would not be a bad idea to always use this CMake flag in the
> >> llvm ports. :)
> > 
> > And this may also fix the issues that all the small
> > memory (aka, RPI*) buliders are facing when trying
> > to do -j4?
> 
> Possibly, as linking is usually the most memory-consuming part of the
> build process (and more so, if debugging is enabled).  Are there build
> logs available somewhere for those RPI builders?
> 

There is a collection of RPI3 buildworld logs in
http://www.zefox.net/~fbsd/rpi3/swaptests/
The more recent experiments are sorted by revision first, then swap config
and then other modifications.

If I can do anything to make the records more useful please let me know.

hth,

bob prohaska
 


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Manfred Antar]

> On Aug 20, 2018, at 10:47 AM, Kyle Evans  wrote:
> 
> On Mon, Aug 20, 2018 at 12:44 PM, Manfred Antar  
> wrote:
>> 
>> 
>>> On Aug 20, 2018, at 9:20 AM, Warner Losh  wrote:
>>> 
>>> 
>>> 
>>> On Mon, Aug 20, 2018 at 9:55 AM, Kyle Evans  wrote:
>>> On Mon, Aug 20, 2018 at 10:39 AM, Manfred Antar  
>>> wrote:
 Hi
 How do i get the menu back after switching to LUA loader?
 I’m using a comconsole. Before the switch to LUA I had a menu and a little 
 devil with pitchfork:)
 Now just get the countdown from 10 , then boot.
 Here is my loader.conf.local:
 
 console="comconsole"
 beastie_disable="NO"
 hw.vga.textmode=1
 kern.vt.spclkeys=15
 nvidia_load="YES"
 linux_common_load="YES"
 linux_load="YES"
 linprocfs_load="YES"
 linsysfs_load=“YES"
 
>>> 
>>> Hmm... seems that there was a miscommunication at some point, and the
>>> menu is disabled explicitly on serial boots. console="comconsole" is
>>> enough to disable this in lualoader land. An excerpt from an e-mail I
>>> was forwarded regarding this:
>>> 
>>> if $console contains (space or comma separated) "efi" then do
>>> not draw a menu (fall back to autoboot routine).
>>> 
>>> if $beastie_disable is set to "YES" (case insensitive), then do not
>>> draw a menu (fall back to autoboot routine).
>>> 
>>> We are clearly doing this wrong. I will fix it ASAP.
>>> 
>>> I think that we need  https://reviews.freebsd.org/D16816 to fix all the 
>>> bits.
>>> 
>>> Warner
>> 
>> Ok that works
>> I had to edit /boot/lua/drawer.lua to get the devil-pitchfork back, 
>> otherwise just have devilhead.
>> 
>> drawer.default_color_logodef = 'orb'
>> drawer.default_bw_logodef = ‘orbbw'
>> 
>> to:
>> 
>> drawer.default_color_logodef = 'beastie'
>> drawer.default_bw_logodef = ‘beastiebw'
>> 
>> and it’s pretty much the same as the 4th menu i had before.
>> Is there a way to put:
>> 
>> drawer.default_color_logodef = 'beastie'
>> drawer.default_bw_logodef = 'beastiebw'
>> 
>> somewhere else ie loader.rc or loader.lua.local so I don’t have to edit the 
>> /boot/lua/drawer.lua everytime I rebuild world?
>> It doesn’t work to put in loader.conf.local
> 
> In loader.conf(5) land, this is called "loader_logo"
> 
> e.g. loader_logo=“beastie"

I got it now.
I was putting the wrong line in loader.conf.local
loader_logo works!!
Thanks

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Kyle Evans]

On Mon, Aug 20, 2018 at 12:44 PM, Manfred Antar  wrote:
>
>
>> On Aug 20, 2018, at 9:20 AM, Warner Losh  wrote:
>>
>>
>>
>> On Mon, Aug 20, 2018 at 9:55 AM, Kyle Evans  wrote:
>> On Mon, Aug 20, 2018 at 10:39 AM, Manfred Antar  
>> wrote:
>> > Hi
>> > How do i get the menu back after switching to LUA loader?
>> > I’m using a comconsole. Before the switch to LUA I had a menu and a little 
>> > devil with pitchfork:)
>> > Now just get the countdown from 10 , then boot.
>> > Here is my loader.conf.local:
>> >
>> > console="comconsole"
>> > beastie_disable="NO"
>> > hw.vga.textmode=1
>> > kern.vt.spclkeys=15
>> > nvidia_load="YES"
>> > linux_common_load="YES"
>> > linux_load="YES"
>> > linprocfs_load="YES"
>> > linsysfs_load=“YES"
>> >
>>
>> Hmm... seems that there was a miscommunication at some point, and the
>> menu is disabled explicitly on serial boots. console="comconsole" is
>> enough to disable this in lualoader land. An excerpt from an e-mail I
>> was forwarded regarding this:
>>
>> if $console contains (space or comma separated) "efi" then do
>> not draw a menu (fall back to autoboot routine).
>>
>> if $beastie_disable is set to "YES" (case insensitive), then do not
>> draw a menu (fall back to autoboot routine).
>>
>> We are clearly doing this wrong. I will fix it ASAP.
>>
>> I think that we need  https://reviews.freebsd.org/D16816 to fix all the bits.
>>
>> Warner
>
> Ok that works
> I had to edit /boot/lua/drawer.lua to get the devil-pitchfork back, otherwise 
> just have devilhead.
>
> drawer.default_color_logodef = 'orb'
> drawer.default_bw_logodef = ‘orbbw'
>
> to:
>
> drawer.default_color_logodef = 'beastie'
> drawer.default_bw_logodef = ‘beastiebw'
>
> and it’s pretty much the same as the 4th menu i had before.
> Is there a way to put:
>
> drawer.default_color_logodef = 'beastie'
> drawer.default_bw_logodef = 'beastiebw'
>
> somewhere else ie loader.rc or loader.lua.local so I don’t have to edit the 
> /boot/lua/drawer.lua everytime I rebuild world?
> It doesn’t work to put in loader.conf.local

In loader.conf(5) land, this is called "loader_logo"

e.g. loader_logo="beastie"

Thanks,

Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Manfred Antar]

> On Aug 20, 2018, at 9:20 AM, Warner Losh  wrote:
> 
> 
> 
> On Mon, Aug 20, 2018 at 9:55 AM, Kyle Evans  wrote:
> On Mon, Aug 20, 2018 at 10:39 AM, Manfred Antar  
> wrote:
> > Hi
> > How do i get the menu back after switching to LUA loader?
> > I’m using a comconsole. Before the switch to LUA I had a menu and a little 
> > devil with pitchfork:)
> > Now just get the countdown from 10 , then boot.
> > Here is my loader.conf.local:
> >
> > console="comconsole"
> > beastie_disable="NO"
> > hw.vga.textmode=1
> > kern.vt.spclkeys=15
> > nvidia_load="YES"
> > linux_common_load="YES"
> > linux_load="YES"
> > linprocfs_load="YES"
> > linsysfs_load=“YES"
> >
> 
> Hmm... seems that there was a miscommunication at some point, and the
> menu is disabled explicitly on serial boots. console="comconsole" is
> enough to disable this in lualoader land. An excerpt from an e-mail I
> was forwarded regarding this:
> 
> if $console contains (space or comma separated) "efi" then do
> not draw a menu (fall back to autoboot routine).
> 
> if $beastie_disable is set to "YES" (case insensitive), then do not
> draw a menu (fall back to autoboot routine).
> 
> We are clearly doing this wrong. I will fix it ASAP.
> 
> I think that we need  https://reviews.freebsd.org/D16816 to fix all the bits.
> 
> Warner 

Ok that works
I had to edit /boot/lua/drawer.lua to get the devil-pitchfork back, otherwise 
just have devilhead.

drawer.default_color_logodef = 'orb'
drawer.default_bw_logodef = ‘orbbw'

to:

drawer.default_color_logodef = 'beastie'
drawer.default_bw_logodef = ‘beastiebw'

and it’s pretty much the same as the 4th menu i had before.
Is there a way to put:

drawer.default_color_logodef = 'beastie'
drawer.default_bw_logodef = 'beastiebw'

somewhere else ie loader.rc or loader.lua.local so I don’t have to edit the 
/boot/lua/drawer.lua everytime I rebuild world?
It doesn’t work to put in loader.conf.local
Thanks
Manfred
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Dimitry Andric]

On 20 Aug 2018, at 16:26, Rodney W. Grimes  
wrote:
> 
>> On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
>>> 
>>> I am running current compiling LLVM60 and when it comes to linking
>>> basically all the processes on my computer gets killed; Chrome, Firefox and
>>> some of the LLVM threads as well
...
> 
>> It is running out of RAM while running multiple parallel link jobs.  If
>> you are building using WITH_DEBUG, turn that off, it consumes large
>> amounts of memory.  If you must have debug info, try adding the
>> following flag to the CMake command line:
>> 
>> -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
>> 
>> That will limit the amount of parallel link jobs to 1, even if you
>> specify -j 8 to gmake or ninja.
>> 
>> Brooks, it would not be a bad idea to always use this CMake flag in the
>> llvm ports. :)
> 
> And this may also fix the issues that all the small
> memory (aka, RPI*) buliders are facing when trying
> to do -j4?

Possibly, as linking is usually the most memory-consuming part of the
build process (and more so, if debugging is enabled).  Are there build
logs available somewhere for those RPI builders?

I have attached a patch for most of the llvm ports, which sets the
LLVM_PARALLEL_LINK_JOBS CMake flag during the configure phase.

-Dimitry


devel__llvm-all-parallel-link-jobs-1.diff
Description: Binary data


signature.asc
Description: Message signed with OpenPGP

Re: LUA boot loader coming very soon

[Warner Losh]

On Mon, Aug 20, 2018 at 9:55 AM, Kyle Evans  wrote:

> On Mon, Aug 20, 2018 at 10:39 AM, Manfred Antar 
> wrote:
> > Hi
> > How do i get the menu back after switching to LUA loader?
> > I’m using a comconsole. Before the switch to LUA I had a menu and a
> little devil with pitchfork:)
> > Now just get the countdown from 10 , then boot.
> > Here is my loader.conf.local:
> >
> > console="comconsole"
> > beastie_disable="NO"
> > hw.vga.textmode=1
> > kern.vt.spclkeys=15
> > nvidia_load="YES"
> > linux_common_load="YES"
> > linux_load="YES"
> > linprocfs_load="YES"
> > linsysfs_load=“YES"
> >
>
> Hmm... seems that there was a miscommunication at some point, and the
> menu is disabled explicitly on serial boots. console="comconsole" is
> enough to disable this in lualoader land. An excerpt from an e-mail I
> was forwarded regarding this:
>
> if $console contains (space or comma separated) "efi" then do
> not draw a menu (fall back to autoboot routine).
>
> if $beastie_disable is set to "YES" (case insensitive), then do not
> draw a menu (fall back to autoboot routine).
>
> We are clearly doing this wrong. I will fix it ASAP.
>

I think that we need  https://reviews.freebsd.org/D16816 to fix all the
bits.

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Kyle Evans]

On Mon, Aug 20, 2018 at 10:39 AM, Manfred Antar  wrote:
> Hi
> How do i get the menu back after switching to LUA loader?
> I’m using a comconsole. Before the switch to LUA I had a menu and a little 
> devil with pitchfork:)
> Now just get the countdown from 10 , then boot.
> Here is my loader.conf.local:
>
> console="comconsole"
> beastie_disable="NO"
> hw.vga.textmode=1
> kern.vt.spclkeys=15
> nvidia_load="YES"
> linux_common_load="YES"
> linux_load="YES"
> linprocfs_load="YES"
> linsysfs_load=“YES"
>

Hmm... seems that there was a miscommunication at some point, and the
menu is disabled explicitly on serial boots. console="comconsole" is
enough to disable this in lualoader land. An excerpt from an e-mail I
was forwarded regarding this:

if $console contains (space or comma separated) "efi" then do
not draw a menu (fall back to autoboot routine).

if $beastie_disable is set to "YES" (case insensitive), then do not
draw a menu (fall back to autoboot routine).

We are clearly doing this wrong. I will fix it ASAP.

Thanks,

Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: LUA boot loader coming very soon

[Manfred Antar]

Hi
How do i get the menu back after switching to LUA loader?
I’m using a comconsole. Before the switch to LUA I had a menu and a little 
devil with pitchfork:)
Now just get the countdown from 10 , then boot.
Here is my loader.conf.local:

console="comconsole"
beastie_disable="NO"
hw.vga.textmode=1
kern.vt.spclkeys=15
nvidia_load="YES"
linux_common_load="YES"
linux_load="YES"
linprocfs_load="YES"
linsysfs_load=“YES"

I also had a loader.rc file but it does not seem relevant to LUA:

\ Loader.rc
\ $FreeBSD: head/stand/i386/loader/loader.rc 332413 2018-04-11 18:02:13Z imp $
\
\ Includes additional commands
include /boot/loader.4th
include /boot/efi.4th
try-include /boot/loader.rc.local

\ Reads and processes loader.conf variables
initialize

maybe-efi-resizecons

\ Tests for password -- executes autoboot first if a password was defined
check-password

\ Load in the boot menu
include /boot/beastie.4th

\ Start the boot menu
beastie-start

Thanks
Manfred
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Fatal trap 12: page fault on Acer Chromebook 720 (peppy)

[Konstantin Belousov]

On Mon, Aug 20, 2018 at 12:45:12AM +0200, Michael Gmelin wrote:
> 
> See here for a screenshot (also including the output of "show pte
> 0xf8000100"):
> 
> https://gist.github.com/grembo/78d0f2a100dd4f16775b85a118769658#file-ddb1-png
It is too early for ddb routines to register.
Ok can you try the following debugging patch, to verify my guess ?

diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c
index 18777d23f09..cd05fdb763f 100644
--- a/sys/amd64/amd64/pmap.c
+++ b/sys/amd64/amd64/pmap.c
@@ -1052,8 +1052,7 @@ create_pagetables(vm_paddr_t *firstaddr)
pd_p = (pd_entry_t *)DMPDkernphys;
for (i = 0; i < (NPDEPG * nkdmpde); i++)
pd_p[i] = (i << PDRSHIFT) | X86_PG_V | PG_PS | pg_g |
-   X86_PG_M | X86_PG_A | pg_nx |
-   bootaddr_rwx(i << PDRSHIFT);
+   X86_PG_M | X86_PG_A | pg_nx | X86_PG_RW;
for (i = 0; i < nkdmpde; i++)
pdp_p[i] = (DMPDkernphys + ptoa(i)) | X86_PG_RW |
X86_PG_V;
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Mark Millard]

Rodney W. Grimes freebsd-rwg at pdx.rh.CN85.dnsmgr.net wrote on
Mon Aug 20 14:26:55 UTC 2018 :

> > On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
> > > 
> > > I am running current compiling LLVM60 and when it comes to linking
> > > basically all the processes on my computer gets killed; Chrome, Firefox 
> > > and
> > > some of the LLVM threads as well
> > ...
> > > llvm/build % ninja -j8
> > > [2408/2473] Building CXX object
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> > > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes -I../lib/Passes
> > > -Iinclude -I../include -isystem /usr/local/include -fPIC
> > > -fvisibility-inlines-hidden -Werror=date-time
> > > -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> > > -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> > > -Wmissing-field-initializers -pedantic -Wno-long-long
> > > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> > > -Wstring-conversion -fdiagnostics-color -g-fno-exceptions -fno-rtti 
> > > -MD
> > > -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> > > ../lib/Passes/PassBuilder.cpp
> > > c++: error: unable to execute command: Killed
> > 
> > It is running out of RAM while running multiple parallel link jobs.  If
> > you are building using WITH_DEBUG, turn that off, it consumes large
> > amounts of memory.  If you must have debug info, try adding the
> > following flag to the CMake command line:
> > 
> > -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> > 
> > That will limit the amount of parallel link jobs to 1, even if you
> > specify -j 8 to gmake or ninja.
> > 
> > Brooks, it would not be a bad idea to always use this CMake flag in the
> > llvm ports. :)
> 
> And this may also fix the issues that all the small
> memory (aka, RPI*) buliders are facing when trying
> to do -j4?

It may help, but:

Even just compiles with no links running can get the kills
in such small system contexts.

And going for a simpler context that can demonstrate
the behavior . . .

Taking a Pine64+ 2GB as an example (4 cores with 1
HW-thread per core, 2 GiBytes of RAM, USB device for
root file system and swap partition):

In another login:
# stress -d 2 -m 4 --vm-keep --vm-bytes 536870912

That "4" and "536870912" total to the 2 GiBytes so
swapping is induced for the context in question.
(Scale --vm-bytes appropriately to context.)
[Note: I had 3 GiBytes of swap space in a partition
for the below.]

[stress is from the port sysutils/stress .]

I had left the default vm.pageout_oom_seq=12 in place for this,
making the kills easier to get than the 120 figure would. It
does not take very long generally for some sort of message to
show up. Sometimes kills happen:

My test environment has Mark Johnston patches to report
things not normally reported:

waited 9s for async swap write
waited 9s for swap buffer
waited 9s for async swap write
waited 9s for async swap write
waited 9s for async swap write
v_free_count: 1357, v_inactive_count: 1
Aug 20 06:04:27 pine64 kernel: pid 1010 (stress), uid 0, was killed: out of 
swap space
waited 5s for async swap write
waited 5s for swap buffer
waited 5s for async swap write
waited 5s for async swap write
waited 5s for async swap write
waited 13s for async swap write
waited 12s for swap buffer
waited 13s for async swap write
waited 12s for async swap write
waited 12s for async swap write
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 161177, size: 131072
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164766, size: 65536
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164064, size: 12288
. . .

(I made multiple runs, most manually stopped. None run for
all that long.)

(Warning: the swap space part of "killed: out of swap space"
can be a misnomer. Killing is driven by having low free RAM
for sufficiently long. vm.pageout_oom_seq controls how
long. Swap space may be unused, little used, or actually be
low. With 3 GiBytes of swap space in the partition, these
runs were not low on swap space.)

Even with fairly stable ms/w figures the queue depths can
get to be large, implying long times before the last queued
write completes. For example (for a USB storage device for
the root file system and swap partition):

dT: 1.006s  w: 1.000s
L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/wd/s   kBps   ms/d   
%busy Name
   0  0  0  00.0  0  00.0  0  00.0
0.0| mmcsd0
  56312  0  00.0312  19985  142.6  0  00.0   
99.6| da0

Large quantities of bytes are being queued in a short time
relative to the around 20 MiByte/sec figure that can be
sustained. The quantity is spread over the queue entries.


Note: The Pine46+ 

Re: What's this gregset_t gregs thing

[blubee blubeeme]

On Mon, Aug 20, 2018 at 9:49 PM Shawn Webb 
wrote:

> On Mon, Aug 20, 2018 at 08:33:04AM +0800, blubee blubeeme wrote:
> > Linux has gregset_t gregs;
> >
> https://github.com/lattera/glibc/blob/master/sysdeps/unix/sysv/linux/x86/sys/ucontext.h
>
> Please note that that repo is hopelessly old and out-of-date. I
> created it as a mirror of the official glibc repo because I was doing
> some offensive research against GNU's RTLD way back in 2011-2012. The
> repo hasn't been updated since.
>
> Thanks,
>
> --
> Shawn Webb
> Cofounder and Security Engineer
> HardenedBSD
>
> Tor-ified Signal:+1 443-546-8752
> Tor+XMPP+OTR:latt...@is.a.hacker.sx
> GPG Key ID:  0x6A84658F52456EEE
> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE
>

Thanks for all the replies, I think OSX also lacks this gregs_t thing as
well I was able to lock it up behind some ifdefs and all is well.

Best,
Owen
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[blubee blubeeme]

On Mon, Aug 20, 2018 at 10:26 PM Rodney W. Grimes <
freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:

> > On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
> > >
> > > I am running current compiling LLVM60 and when it comes to linking
> > > basically all the processes on my computer gets killed; Chrome,
> Firefox and
> > > some of the LLVM threads as well
> > ...
> > > llvm/build % ninja -j8
> > > [2408/2473] Building CXX object
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > > /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> > > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes
> -I../lib/Passes
> > > -Iinclude -I../include -isystem /usr/local/include -fPIC
> > > -fvisibility-inlines-hidden -Werror=date-time
> > > -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> > > -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> > > -Wmissing-field-initializers -pedantic -Wno-long-long
> > > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> > > -Wstring-conversion -fdiagnostics-color -g-fno-exceptions
> -fno-rtti -MD
> > > -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> > > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> > > ../lib/Passes/PassBuilder.cpp
> > > c++: error: unable to execute command: Killed
> >
> > It is running out of RAM while running multiple parallel link jobs.  If
> > you are building using WITH_DEBUG, turn that off, it consumes large
> > amounts of memory.  If you must have debug info, try adding the
> > following flag to the CMake command line:
> >
> > -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> >
> > That will limit the amount of parallel link jobs to 1, even if you
> > specify -j 8 to gmake or ninja.
> >
> > Brooks, it would not be a bad idea to always use this CMake flag in the
> > llvm ports. :)
>
> And this may also fix the issues that all the small
> memory (aka, RPI*) buliders are facing when trying
> to do -j4?
>
>
> --
> Rod Grimes
> rgri...@freebsd.org


Someone mentioned earlier that debug builds cause a lot of memory usage and
it's true that I am building with debug.
I couldn't grab the relevant text from top but the posted was after
everything settled down.
The memory situation got so bad that xserver died and I was booted to
terminal.

I saw ld.lld using up to 12GB of ram for each process, that will definitely
use up 32GB of ram.

I can't avoid building w/ debug symbols since I have to debug the program
are there any other options other than
building with only 1 thread? Reason being even when I set gmake -j1 the
problem comes just a lot later during linking
stage.

Best,
Owen
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Rodney W. Grimes]

> On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
> > 
> > I am running current compiling LLVM60 and when it comes to linking
> > basically all the processes on my computer gets killed; Chrome, Firefox and
> > some of the LLVM threads as well
> ...
> > llvm/build % ninja -j8
> > [2408/2473] Building CXX object
> > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> > /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes -I../lib/Passes
> > -Iinclude -I../include -isystem /usr/local/include -fPIC
> > -fvisibility-inlines-hidden -Werror=date-time
> > -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> > -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> > -Wmissing-field-initializers -pedantic -Wno-long-long
> > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> > -Wstring-conversion -fdiagnostics-color -g-fno-exceptions -fno-rtti -MD
> > -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> > lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> > ../lib/Passes/PassBuilder.cpp
> > c++: error: unable to execute command: Killed
> 
> It is running out of RAM while running multiple parallel link jobs.  If
> you are building using WITH_DEBUG, turn that off, it consumes large
> amounts of memory.  If you must have debug info, try adding the
> following flag to the CMake command line:
> 
> -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> 
> That will limit the amount of parallel link jobs to 1, even if you
> specify -j 8 to gmake or ninja.
> 
> Brooks, it would not be a bad idea to always use this CMake flag in the
> llvm ports. :)

And this may also fix the issues that all the small
memory (aka, RPI*) buliders are facing when trying
to do -j4?


-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Loading carp module crash i386 (12-ALPHA2), seems VNET related

[Olivier Cochard-Labbé]

Just loading carp kernel module is enough to panic it:

[root@router]~# uname -a
FreeBSD router.bsdrp.net 12.0-ALPHA2 FreeBSD 12.0-ALPHA2  r338100M  i386
[root@router]~# kldload carp
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x14de0f4c
fault code  = supervisor write, page not present
instruction pointer = 0x20:0x1201657c
stack pointer   = 0x28:0x11f3e804
frame pointer   = 0x28:0x11f3e80c
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 55291 (kldload)
[ thread pid 55291 tid 100070 ]
Stopped at  vnet_carpstats_init+0x2c:   movl
%eax,__stop_set_vnet(%ecx,%esi,1)

A backtrace on kgdb gives this output:

(kgdb) bt
#0  __curthread () at ./machine/pcpu.h:226
#1  doadump (textdump=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_shutdown.c:366
#2  0x009f69ee in db_dump (dummy=302081404, dummy2=false, dummy3=-1,
dummy4=0x11f3e4fc "") at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:574
#3  0x009f67c7 in db_command (last_cmdp=,
cmd_table=, dopager=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:481
#4  0x009f6520 in db_command_loop () at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:534
#5  0x009f978d in db_trap (type=12, code=0) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_main.c:252
#6  0x0113e29e in kdb_trap (type=12, code=0, tf=0x11f3e7c4) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/subr_kdb.c:693
#7  0x016d593c in trap_fatal (frame=0x11f3e7c4, eva=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:989
#8  0x016d5a43 in trap_pfault (frame=, usermode=0,
eva=350097228) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:827
#9  0x016d508f in trap (frame=0x11f3e7c4) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:519
#10 0xffc0315d in ?? ()
#11 0x11f3e7c4 in ?? ()
#12 0x0121c7ef in vnet_register_sysinit (arg=0x120182cc
) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/net/vnet.c:500
#13 0x010c18f0 in linker_file_sysinit (lf=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:236
#14 linker_load_file (filename=, result=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:462
#15 linker_load_module (kldname=, modname=0x11449c00 "carp",
parent=0x0, verinfo=0x0, lfpp=0x11f3ea6c) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:2092
#16 0x010c30d1 in kern_kldload (td=0x1157c380, file=0x11449c00 "carp",
fileid=0x11f3ea98) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:1071
#17 0x010c31ee in sys_kldload (td=0x1157c380, uap=0x1157c604) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:1097
#18 0x016d6217 in syscallenter (td=) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/../../kern/subr_syscall.c:135
#19 syscall (frame=0x11f3eba8) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:1152
#20 0xffc033a7 in ?? ()
#21 0x11f3eba8 in ?? ()
Backtrace stopped: Cannot access memory at address 0xffbfecec
(kgdb) frame 12
#12 0x0121c7ef in vnet_register_sysinit (arg=0x120182cc
) at
/usr/local/BSDRP/TESTING/FreeBSD/src/sys/net/vnet.c:500
500 vs->func(vs->arg);


Regards,

Olivier
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: What's this gregset_t gregs thing

[Shawn Webb]

On Mon, Aug 20, 2018 at 08:33:04AM +0800, blubee blubeeme wrote:
> Linux has gregset_t gregs;
> https://github.com/lattera/glibc/blob/master/sysdeps/unix/sysv/linux/x86/sys/ucontext.h

Please note that that repo is hopelessly old and out-of-date. I
created it as a mirror of the official glibc repo because I was doing
some offensive research against GNU's RTLD way back in 2011-2012. The
repo hasn't been updated since.

Thanks,

-- 
Shawn Webb
Cofounder and Security Engineer
HardenedBSD

Tor-ified Signal:+1 443-546-8752
Tor+XMPP+OTR:latt...@is.a.hacker.sx
GPG Key ID:  0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE


signature.asc
Description: PGP signature

Re: Sharing compiled builds between multiple 12-CURRENT boxes.

[Cy Schubert]

In message <20180820062511.yaa2rimgmwpv3hxx@kazhap>, Dhananjay Balan 
writes:
> On Sat, Aug 18, 2018 at 04:36:15PM -0700, Cy Schubert wrote:
> > You can use NFS or rsync. Make sure the src and obj path names are 
> > exactly the same on all thes servers. I used to use NFS until a year or 
> > two ago when I started using different patches on different machines. 
> > On occasion I've used rsync followed by a NO_CLEAN, WORLDFAST, KERNFAST 
> > build on the target machine or simply installkernel and installworld.
> > 
>
> I had a quick try of this, with rsync. installkernel worked properly
> and I can boot, howver installworld fails with lot of $PROGRAM not
> found erroer for env, mktemp etc.
>
> Thanks for all the suggestions I will grok at the installworld a bit
> more, and will also give PkgBase a try.

Make sure the archs are the same. Make sure the src and obj pathnames 
on both servers are exactly the same. Make sure /usr/src (or wherever 
you put it) is the same, it too needs to be rsynced.

The tiniest differences will lead to this error.

Don't do this _during_ a build on the source machine, and other common 
sense things.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  http://www.FreeBSD.org

The need of the many outweighs the greed of the few.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: What's this gregset_t gregs thing

[Jan Beich]

blubee blubeeme  writes:

> Linux has gregset_t gregs;
> https://github.com/lattera/glibc/blob/master/sysdeps/unix/sysv/linux/x86/sys/ucontext.h
>
> Defined above, I also see it in the RISC-V glibc stuff as well.
>
> FreeBSD doesn't seem to have this field defined, I see FreeBSD uses
> /usr/include/x86/ucontext.h
>
> but there's no gregs;

__gregs exists on FreeBSD arm*, inherited from NetBSD.

> If I wanted to return mcontext.gregs
>
> How could I implement that on FreeBSD?

Only as gregset_t? If not return registers individually e.g.,
https://searchfox.org/mozilla-central/rev/83562422ecb0f5683da7a9a26ce05ce62bc0c882/js/src/wasm/WasmSignalHandlers.cpp#244
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: building LLVM threads gets killed

[Dimitry Andric]

On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
> 
> I am running current compiling LLVM60 and when it comes to linking
> basically all the processes on my computer gets killed; Chrome, Firefox and
> some of the LLVM threads as well
...
> llvm/build % ninja -j8
> [2408/2473] Building CXX object
> lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> FAILED: lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o
> /usr/bin/c++  -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS
> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Passes -I../lib/Passes
> -Iinclude -I../include -isystem /usr/local/include -fPIC
> -fvisibility-inlines-hidden -Werror=date-time
> -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra
> -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> -Wmissing-field-initializers -pedantic -Wno-long-long
> -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -Wstring-conversion -fdiagnostics-color -g-fno-exceptions -fno-rtti -MD
> -MT lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -MF
> lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o.d -o
> lib/Passes/CMakeFiles/LLVMPasses.dir/PassBuilder.cpp.o -c
> ../lib/Passes/PassBuilder.cpp
> c++: error: unable to execute command: Killed

It is running out of RAM while running multiple parallel link jobs.  If
you are building using WITH_DEBUG, turn that off, it consumes large
amounts of memory.  If you must have debug info, try adding the
following flag to the CMake command line:

-D LLVM_PARALLEL_LINK_JOBS:STRING="1"

That will limit the amount of parallel link jobs to 1, even if you
specify -j 8 to gmake or ninja.

Brooks, it would not be a bad idea to always use this CMake flag in the
llvm ports. :)

-Dimitry



signature.asc
Description: Message signed with OpenPGP

Re: building LLVM threads gets killed

[Mark Millard]

blubee blubeeme gurenchan at gmail.com wrote on
Mon Aug 20 03:02:01 UTC 2018 :

> I am running current compiling LLVM60 and when it comes to linking
> basically all the processes on my computer gets killed; Chrome, Firefox and
> some of the LLVM threads as well

> . . .

> last pid: 20965;  load averages:  0.64,  5.79,  7.73
>  up 12+01:35:46  11:00:36
> 76 processes:  1 running, 75 sleeping
> CPU:  0.8% user,  0.5% nice,  1.0% system,  0.0% interrupt, 98.1% idle
> Mem: 10G Active, 3G Inact, 100M Laundry, 13G Wired, 6G Free
> ARC: 4G Total, 942M MFU, 1G MRU, 1M Anon, 43M Header, 2G Other
>  630M Compressed, 2G Uncompressed, 2.74:1 Ratio
> Swap: 2G Total, 1G Used, 739M Free, 63% Inuse
> . . .

The timing of that top output relative to the first or
any OOM kill of a process is not clear. After? Just
before? How long before? What it is like leading up
to the first kill is of interest.

Folks that deal with this are likely to want do know
if you got console messages ( or var/log/messages content)
such as:

pid 49735 (c++), uid 0, was killed: out of swap space

(Note: "out of swap space" can be a misnomer for having
low Free RAM for "too long" [vm.pageout_oom_seq based],
even with swap unused or little used.)

And: Were you also getting messages like:

swap_pager_getswapspace(4): failed

and/or:

swap_pager: out of swap space

(These indicate the "killed: out of swap space" is not
necessarily a misnomer relative to swap space, even if
low free RAM over a time drives the process kills.)

How about messages like:

swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28139, size: 65536

or any I/O error reports or retry reports?



Notes:

Mark Johnston published a patch used for some investigations of
the OOM killing:

https://people.freebsd.org/~markj/patches/slow_swap.diff

But this is tied to the I/O swap latencies involved and if they
are driving some time frames. It just adds more reporting to
the console ( and /var/log/messages ). It is not a fix. IT may
not be likely to report much for your context.


vm.pageout_oom_seq controls the "how long is low free RAM
tolerated" (my hprasing), though the units are not directly
time. In various arm contexts with small boards going from
the default of 12 to 120 allowed things to complete or get
much farther. So:

sysctl vm.pageout_oom_seq=120

but 120 is not the limit: it is a C int parameter.

I'll note that "low free RAM" is as FreeBSD classifies it,
whatever the details are.



Most of the arm examples have been small memory contexts
and many of them likely avoid ZFS and use UFS instead.
ZFS and its ARC and such an additional complicated
context to the type of issue. There are lots of reports
around of the ARC growing too big. I do not know the
status of -r336196 relative to ZFS/ARC memory management
or if more recent versions have improvements. (I do not
use ZFS normally.) I've seen messages making suggestions
for controlling the growth but I'm no ZFS expert.


Just to give an idea what is sufficient to build
devel/llvm60:

I will note that on a Pine64+ 2GB (so 2 GiBytes of RAM
in a aarch64 context with 4 cores, 1 HW-thread per core)
running -r337400, and using UFS on a USB drive and a
swap partition that drive too, I have built devel/llvm60
2 times via poudriere-devel: just one builder
allowed but it being allowed to use all 4 cores in
parallel, about 14.5 hr each time. (Different USB media
each time.) This did require the:

sysctl vm.pageout_oom_seq=120

Mark Johnston's slow_swap.diff patch code did not
report any I/O latency problems in the swap subsystem.

I've also built lang/gcc8 2 times, about 12.5 hrs
each time.

No ZFS, no ARC, no Chrome, no FireFox. Nothing else
major going on beyond the devel/llvm60 build (or, later,
the lang/gcc8 build) in each case.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"