Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty 
> > [Creating objdir /usr/obj/usr/obj/root/git/freebsd/sys/GENERIC...]
> Wrong^
> 
> Note we have 'cd /usr/obj/' and 'MAKEOBJDIRPREFIX=/usr/obj' in
> there, so we get a nested /usr/obj/.CURDIR problem of /usr/obj/usr/obj.

The following would probably help that case:

Index: auto.obj.mk
===
RCS file: /volume/cvs/sjg/mk/auto.obj.mk,v
retrieving revision 1.13
diff -u -p -r1.13 auto.obj.mk
--- auto.obj.mk 24 Mar 2017 20:53:22 -  1.13
+++ auto.obj.mk 18 Apr 2017 22:25:08 -
@@ -41,6 +41,10 @@ MKOBJDIRS= auto
 # Use __objdir here so it is easier to tweak without impacting
 # the logic.
 .if !empty(MAKEOBJDIRPREFIX)
+.if ${.CURDIR:M${MAKEOBJDIRPREFIX}/*} != ""
+# we are already in obj tree!
+__objdir?= ${.CURDIR}
+.endif
 __objdir?= ${MAKEOBJDIRPREFIX}${.CURDIR}
 .endif
 __objdir?= ${MAKEOBJDIR:Uobj}
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty 
Bryan Drewery  wrote:
> > What is the issue above? diff?
> 
> I don't know what the issue is with buildkernel specifically, I never
> looked into it. Buildworld had other issues like rescue/rescue not being
> AUTO_OBJ safe. That's fixed. I forget the details of buildworld as well.
> One of the simpler changes is to remove the 'make obj' treewalk and
> prefer .OBJDIR/OBJTOP over other ${CANONICALOBJDIR} ${OBJTREE} type
> hacks. My patches are quite mangled with the idea of moving to
> /usr/obj/CURDIR/TARGET.TARGET_ARCH/RELDIR and I need to split it all up
> still.

Ok

> 
> > 
> > FWIW the latest version of auto.obj.mk deals with __objdir=obj which is
> > a corner case compared to any use of MAKEOJDIRPREFIX etc.
> > 
> 
> It does not fix the issues.

Not this lot no.  Its for folk wanting to use the traditional
${.CURDIR}/obj/ which used to throw an error.
I haven't used ${.CURDIR}/obj/ in almost 20 years ;-)


> >  SIZE="size"  INSTALL="sh /root/git/freebsd/tools/install.sh"  
> > PATH=/usr/obj/root/git/freebsd/tmp/legacy/usr/sbin:/usr/obj/root/git/freebsd/tmp/legacy/usr/bin:/usr/obj/root/git/freebsd/tmp/legacy/bin:/usr/obj/root/git/freebsd/tmp/usr/sbin:/usr/obj/root/git/freebsd/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin
> >  make  -j 1
> > 5 -D NO_FILEMON -d M -J 15,16 -m /root/git/freebsd/share/mk  KERNEL=kernel 
> > obj
> > [Creating objdir /usr/obj/usr/obj/root/git/freebsd/sys/GENERIC...]
> Wrong^
> 
> Note we have 'cd /usr/obj/' and 'MAKEOBJDIRPREFIX=/usr/obj' in
> there, so we get a nested /usr/obj/.CURDIR problem of /usr/obj/usr/obj.

Yes, ugly.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildkernel broken for META_MODE

2017-04-18 Thread Bryan Drewery 
On 4/18/2017 2:33 PM, Simon J. Gerraty wrote:
> Bryan Drewery  wrote:
>> Aha /usr/obj/usr/obj.
>>
>> That was in Renato's report as well.
>>
>> The bug is WITH_AUTO_OBJ. I just confirmed that. A bunch of errors occur
>> when doing the first build and the opt_*.h files are not generated in
>> the "proper" place by config(8).
>>
>> WITH_AUTO_OBJ is not really workable yet in buildworld/buildkernel. I
>> have patches that I will commit soon to add support for them (meaning to
>> work as it all does not but without a 'make obj' tree walk).
> 
> What is the issue above? diff?

I don't know what the issue is with buildkernel specifically, I never
looked into it. Buildworld had other issues like rescue/rescue not being
AUTO_OBJ safe. That's fixed. I forget the details of buildworld as well.
One of the simpler changes is to remove the 'make obj' treewalk and
prefer .OBJDIR/OBJTOP over other ${CANONICALOBJDIR} ${OBJTREE} type
hacks. My patches are quite mangled with the idea of moving to
/usr/obj/CURDIR/TARGET.TARGET_ARCH/RELDIR and I need to split it all up
still.

> 
> FWIW the latest version of auto.obj.mk deals with __objdir=obj which is
> a corner case compared to any use of MAKEOJDIRPREFIX etc.
> 

It does not fix the issues.

> --
 stage 2.2: rebuilding the object tree
> --
> cd /usr/obj/root/git/freebsd/sys/GENERIC; COMPILER_VERSION=4  
> COMPILER_FEATURES=c++11  COMPILER_TYPE=clang  
> COMPILER_FREEBSD_VERSION=126 MAKEOBJDIRPREFIX=/usr/obj  
> MACHINE_ARCH=amd64  MACHINE=amd64  CPUTYPE= BUILD_TOOLS_META=.NOMETA 
> GROFF_BIN_PATH=/usr/obj/root/git/freebsd/tmp/legacy/usr/bin  GROFF_FONT_PATH
> =/usr/obj/root/git/freebsd/tmp/legacy/usr/share/groff_font  
> GROFF_TMAC_PATH=/usr/obj/root/git/freebsd/tmp/legacy/usr/share/tmac 
> CC="/usr/local/bin/ccache cc -target x86_64-unknown-freebsd12.0 
> --sysroot=/usr/obj/root/git/freebsd/tmp 
> -B/usr/obj/root/git/freebsd/tmp/usr/bin" CXX="/usr/local/bin/ccache c++  
> -target x86_
> 64-unknown-freebsd12.0 --sysroot=/usr/obj/root/git/freebsd/tmp 
> -B/usr/obj/root/git/freebsd/tmp/usr/bin"  CPP="cpp -target 
> x86_64-unknown-freebsd12.0 --sysroot=/usr/obj/root/git/freebsd/tmp 
> -B/usr/obj/root/git/freebsd/tmp/usr/bin"  AS="as" AR="ar" LD="ld" 
> LLVM_LINK=""  NM=nm OBJCOPY="objcopy"  RANLIB=ranlib STRINGS=
>  SIZE="size"  INSTALL="sh /root/git/freebsd/tools/install.sh"  
> PATH=/usr/obj/root/git/freebsd/tmp/legacy/usr/sbin:/usr/obj/root/git/freebsd/tmp/legacy/usr/bin:/usr/obj/root/git/freebsd/tmp/legacy/bin:/usr/obj/root/git/freebsd/tmp/usr/sbin:/usr/obj/root/git/freebsd/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin
>  make  -j 1
> 5 -D NO_FILEMON -d M -J 15,16 -m /root/git/freebsd/share/mk  KERNEL=kernel obj
> [Creating objdir /usr/obj/usr/obj/root/git/freebsd/sys/GENERIC...]
Wrong^

Note we have 'cd /usr/obj/' and 'MAKEOBJDIRPREFIX=/usr/obj' in
there, so we get a nested /usr/obj/.CURDIR problem of /usr/obj/usr/obj.


> grep: opt_md.h: No such file or directory
> grep: opt_ddb.h: No such file or directory
> grep: opt_kdtrace.h: No such file or directory
> grep: opt_hwpmc_hooks.h: No such file or directory
Wrong^


I will look into this as part of my larger patch, but it's likely due to
the first bug.



-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: https://reviews.freebsd.org/D10232

2017-04-18 Thread Ian Lepore 
On Tue, 2017-04-18 at 23:15 +0300, Toomas Soome wrote:
> Hi!
> 
> The network code rework still needs good people to test out the work,
> especially the non-x86 platforms:)  the url is still: https://reviews
> .freebsd.org/D10232
> 
> Once it checks out, we can go for next phase and get some boost for
> network loading.
> 
> thanks,
> toomas

I use netbooting in loader(8) extensively for armv6, and in fact that's
what I'm knee-deep in for $work today too, so I'll give D10232 a look
tonight or tomorrow at the latest.

-- Ian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty 
Bryan Drewery  wrote:
> Aha /usr/obj/usr/obj.
> 
> That was in Renato's report as well.
> 
> The bug is WITH_AUTO_OBJ. I just confirmed that. A bunch of errors occur
> when doing the first build and the opt_*.h files are not generated in
> the "proper" place by config(8).
> 
> WITH_AUTO_OBJ is not really workable yet in buildworld/buildkernel. I
> have patches that I will commit soon to add support for them (meaning to
> work as it all does not but without a 'make obj' tree walk).

What is the issue above? diff?

FWIW the latest version of auto.obj.mk deals with __objdir=obj which is
a corner case compared to any use of MAKEOJDIRPREFIX etc.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

https://reviews.freebsd.org/D10232

2017-04-18 Thread Toomas Soome 
Hi!

The network code rework still needs good people to test out the work, 
especially the non-x86 platforms:)  the url is still: 
https://reviews.freebsd.org/D10232

Once it checks out, we can go for next phase and get some boost for network 
loading.

thanks,
toomas
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildkernel broken for META_MODE

2017-04-18 Thread Bryan Drewery 
On 4/10/2017 11:14 AM, Bryan Drewery wrote:
> On 4/7/2017 12:13 PM, Ngie Cooper (yaneurabeya) wrote:
>> Hi,
>>  I ran into this error when trying to run a meta mode build (for the 
>> first time). It might be related to the recent assym* ordering changes.
>> Thanks!
>> -Ngie
>>
>> $ cat /etc/src-env.conf
>> WITH_AUTO_OBJ=  yes

Oh I missed this part^

>> WITH_META_MODE= yes
>> UPDATE_DEPENDFILE=  yes

By the way UPDATE_DEPENDFILE is not really a user variable and setting
it may break things.

>> $ pwd
>> /usr/src
>> $ svnversion
>> 316603M
>> $ svn st | grep -v \?
>> M   usr.bin/grep/tests/Makefile
>> $ env SRCCONF=/dev/null NO_FILEMON=1 script ~/bk.ts make buildkernel -j3
>> Script started on Fri Apr  7 11:52:38 2017
>> Command: time make buildkernel -j3
>> --- buildkernel ---
>> make[1]: "/usr/src/Makefile.inc1" line 146: SYSTEM_COMPILER: Determined that 
>> CC=cc matches the source tree.  Not bootstrapping a cross-compiler.
>> --- buildkernel ---
>> --
> Kernel build for GENERIC started on Fri Apr  7 11:52:39 PDT 2017
>> --
>> ===> GENERIC
>> mkdir -p /usr/obj/usr/src/sys
>> --
> stage 1: configuring the kernel
>> --
>> ...
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/acpi_quirks.h

Aha /usr/obj/usr/obj.

That was in Renato's report as well.

The bug is WITH_AUTO_OBJ. I just confirmed that. A bunch of errors occur
when doing the first build and the opt_*.h files are not generated in
the "proper" place by config(8).

WITH_AUTO_OBJ is not really workable yet in buildworld/buildkernel. I
have patches that I will commit soon to add support for them (meaning to
work as it all does not but without a 'make obj' tree walk).


>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/feeder_eq_gen.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/feeder_rate_gen.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/snd_fxdiv_gen.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/miidevs.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/pccarddevs.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/teken_state.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/usbdevs.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/vnode_if.h
>> Building /usr/obj/usr/obj/usr/src/sys/GENERIC/ia32_genassym.o
>> --- ia32_genassym.o ---
>> :1:10: fatal error: 'opt_global.h' file not found
>> #include "opt_global.h"
>>  ^~
>> 1 error generated.
>> *** [ia32_genassym.o] Error code 1
>>
>> make[2]: stopped in /usr/obj/usr/src/sys/GENERIC
>> .ERROR_TARGET='ia32_genassym.o'
>> .ERROR_META_FILE='/usr/obj/usr/obj/usr/src/sys/GENERIC/ia32_genassym.o.meta'
>>
> 
> It was previously reported here too
> https://lists.freebsd.org/pipermail/freebsd-current/2016-June/061946.html.
>  It's a race.  I had forgotten about it but will look at it again.
> 

-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: Sendmail eats CPU on r317039 [after -r316874 it may be -r316951 and -r316973 are not enough to fix everything]

2017-04-18 Thread Mark Millard 
Ronald Klop ronald-lists at klop.ws wrote on Tue Apr 18 09:59:50 UTC 2017:

> there is a thread ono this list about a problem in syslogd which made  
> syslog-clients (like sendmail busy-looping on logging.
> That might be related to this. (it is fixed in the source already, so  
> upgrading again might help)
> See the thread with subject like 'Re: r316958: booting a server takes >10  
> minutes!'
> 
> Regards,
> 
> Ronald.


Yes. But Tom V.'s report is for -r317039, which is after the reported
fixes as far as I can tell. Something besides syslogd might also cause
problems?

In my nearly-default -r317015 ardm64 context [as a VirtualBox guest]
I've not seen the problem, where I did before. (The only reason sendmail
runs in my context is for the messages FreeBSD sends to it own local
accounts. I do not otherwise use mail in this context.)

Tom V.'s report vs. others finding lack of a problem suggests that the
coverage of the fixes is incomplete somehow but useful. I happen to not
be doing whatever causes the problem to appear. I've no clue what might
be different or unusual in Tom V.'s context.

There is also the possibility that Tom V.'s report is a fully independent
issue. But such does not seem all that likely on the initial information.


> On 2017-Apr-17, at 7:57 AM, Mark Millard  wrote:
> 
>> Just an FYI of a more recent report of runaway sendmail on a
>> more recent system version ( -r317039 ):
>> 
>> Begin forwarded message:
>> 
>>> From: Tom Vijlbrief 
>>> Subject: Sendmail eats CPU on r317039
>>> Date: April 17, 2017 at 3:39:37 AM PDT
>>> To: "freebsd-current at freebsd.org" , 
>>> freebsd-arm 
>>> 
>>> On a recent kernel sendmail is constantly consuming CPU.
>>> 
>>> truss -p PID
>>> 
>>> shows:
>>> 
>>> sendto(3,"<22>Apr 17 10:30:33 sendmail[362"...,163,0x0,NULL,0) ERR#55 'No
>>> buffer space available'
>>> nanosleep({ 0.01000 }) = 0 (0x0)
>>> sendto(3,"<22>Apr 17 10:30:33 sendmail[362"...,163,0x0,NULL,0) ERR#55 'No
>>> buffer space available'
>>> nanosleep({ 0.01000 })
>>> ...
>>> 
>>> This is on an arm64 system
>> 
>> Analysis of Tom V.'s context for this may be required.

===
Mark Millard
markmi at dsl-only.net

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Fwd: Sendmail eats CPU on r317039 [after -r316874 it may be -r316951 and -r316973 are not enough to fix everything]

2017-04-18 Thread Ronald Klop 
there is a thread ono this list about a problem in syslogd which made  
syslog-clients (like sendmail busy-looping on logging.
That might be related to this. (it is fixed in the source already, so  
upgrading again might help)
See the thread with subject like 'Re: r316958: booting a server takes >10  
minutes!'


Regards,

Ronald.



On Mon, 17 Apr 2017 16:57:39 +0200, Mark Millard   
wrote:



Just an FYI of a more recent report of runaway sendmail on a
more recent system version ( -r317039 ):

Begin forwarded message:


From: Tom Vijlbrief 
Subject: Sendmail eats CPU on r317039
Date: April 17, 2017 at 3:39:37 AM PDT
To: "freebsd-current at freebsd.org" ,  
freebsd-arm 


On a recent kernel sendmail is constantly consuming CPU.

truss -p PID

shows:

sendto(3,"<22>Apr 17 10:30:33 sendmail[362"...,163,0x0,NULL,0) ERR#55  
'No

buffer space available'
nanosleep({ 0.01000 }) = 0 (0x0)
sendto(3,"<22>Apr 17 10:30:33 sendmail[362"...,163,0x0,NULL,0) ERR#55  
'No

buffer space available'
nanosleep({ 0.01000 })
...

This is on an arm64 system


Analysis of Tom V.'s context for this may be required.

===
Mark Millard
markmi at dsl-only.net

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to  
"freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"