Re: buildworld fails while building static clang library

[Aijaz Baig]

Hi Ed

Yes how do I get those logs? I will be checking dmesg from time to time (is
there a way to tail it by the way??) but then how do I enable verbose
logging while building world? I tried finding out the same by searching on
the internet and by reading the manual page for make.conf but couldn't
figure it out

Is it even possible to have a verbose output?

On Mon, Aug 14, 2017 at 7:36 PM, Ed Maste  wrote:

> On 7 August 2017 at 00:32, Aijaz Baig  wrote:
> > That was some pretty relevant information Ed. Thanks.
>
> Even though it's not a direct cause of the problem you encountered I
> wanted to make sure a there was comprehensive reply to Dimitry's
> question.
>
> > Nonetheless, as I have indicated in my previous email, I hit an error
> quite
> > late in the process now (stage 4.3) when it is apparently linking
> h_raw.o
> > into h_raw.full ( is this linking by the way? what is the .full file
> > there??)
>
> Yes, this is a link step - the .full file is a binary or library with
> debug data included, which is then split into a .debug file containing
> only the debug data and the binary or library with no extension.
>
> There's no indication of what the error is in your log excerpt; are
> there any messages in dmesg from the time of the build?
>



-- 

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Ed Maste]

On 7 August 2017 at 00:32, Aijaz Baig  wrote:
> That was some pretty relevant information Ed. Thanks.

Even though it's not a direct cause of the problem you encountered I
wanted to make sure a there was comprehensive reply to Dimitry's
question.

> Nonetheless, as I have indicated in my previous email, I hit an error quite
> late in the process now (stage 4.3) when it is apparently linking  h_raw.o
> into h_raw.full ( is this linking by the way? what is the .full file
> there??)

Yes, this is a link step - the .full file is a binary or library with
debug data included, which is then split into a .debug file containing
only the debug data and the binary or library with no extension.

There's no indication of what the error is in your log excerpt; are
there any messages in dmesg from the time of the build?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Aijaz Baig]

That was some pretty relevant information Ed. Thanks.

However upon bumping up my RAM, I don't hit this error anymore perhaps I
believe since the relatively large amount of RAM does not necessitate that
much of swap space.

Nonetheless, as I have indicated in my previous email, I hit an error quite
late in the process now (stage 4.3) when it is apparently linking  h_raw.o
into h_raw.full ( is this linking by the way? what is the .full file
there??)

Keen to hear

On Sun, Aug 6, 2017 at 3:04 PM, Ed Maste  wrote:

> On 5 August 2017 at 16:16, Dimitry Andric  wrote:
> >
> > I remember there being an issue with ar and/or ranlib choking when the
> > .a files become too big.  Ed, does that ring any bells?
>
> Our ar (and ranlib, which is the same binary) will produce a corrupt
> symbol table if the .a archive output is larger than 4GB, because we
> support only 32-bit offsets in the older "/" symbol table format, not
> the "/SYM64/" format and 64-bit offsets.
>
> As with GNU ar from binutils 2.17.50 we silently truncate if the
> offset does not fit in 32 bits. I'll have a patch for review soon to
> exit on error rather than produce corrupted output, and hope to look
> at adding /SYM64/ support later on.
>



-- 

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Ed Maste]

On 5 August 2017 at 16:16, Dimitry Andric  wrote:
>
> I remember there being an issue with ar and/or ranlib choking when the
> .a files become too big.  Ed, does that ring any bells?

Our ar (and ranlib, which is the same binary) will produce a corrupt
symbol table if the .a archive output is larger than 4GB, because we
support only 32-bit offsets in the older "/" symbol table format, not
the "/SYM64/" format and 64-bit offsets.

As with GNU ar from binutils 2.17.50 we silently truncate if the
offset does not fit in 32 bits. I'll have a patch for review soon to
exit on error rather than produce corrupted output, and hope to look
at adding /SYM64/ support later on.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Aijaz Baig]

I did notice some swap related messages in dmesg earlier so this time I
bumped up my RAM to 4.25GB (did I tell you I'm running this on a VM?). In
addition I skipped parallel make jobs altogether keeping other things the
same.

So this time around it went a lot further in fact all the way to step4.3
(building everything)

and now it fails at this step:

===> lib/libc/tests/ssp (all)
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_fgets  NO_SUBDIR=1
make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSING
_PROGS=t  PROG=h_fgets )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_gets  NO_SUBDIR=1
make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSING_
PROGS=t  PROG=h_gets )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_getcwd
 NO_SUBDIR=1 make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSIN
G_PROGS=t  PROG=h_getcwd )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_memcpy
 NO_SUBDIR=1 make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSIN
G_PROGS=t  PROG=h_memcpy )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_memmove
 NO_SUBDIR=1 make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSI
NG_PROGS=t  PROG=h_memmove )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_memset
 NO_SUBDIR=1 make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSIN
G_PROGS=t  PROG=h_memset )
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_raw  NO_SUBDIR=1
make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSING_PROGS=t  PROG=h_raw
)
cc -target x86_64-unknown-freebsd12.0 --sysroot=/mnt/obj/usr/src/tmp
-B/mnt/obj/usr/src/tmp/usr/bin -O0 -pipe -march=core2 -fstack-protector-all
-Wstack-protector -fsanitize=bounds -g3 -g -std=gnu99
-fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k
-Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int
-Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value
-Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion
-Wno-unused-local-typedef -Wno-address-of-packed-member -Wno-switch
-Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments  -o
h_raw.full h_raw.o
*** Error code 1

Stop.
make[7]: stopped in /usr/src/lib/libc/tests/ssp
*** Error code 1

Stop.
make[6]: stopped in /usr/src/lib/libc/tests/ssp
*** Error code 1

Stop.
make[5]: stopped in /usr/src/lib/libc/tests
*** Error code 1

Stop.
make[4]: stopped in /usr/src/lib/libc
*** Error code 1

Stop.
make[3]: stopped in /usr/src/lib
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src

keen to hear

On Sun, Aug 6, 2017 at 9:53 AM, Aijaz Baig  wrote:

> Hello
>
> Yes guilty as charged!!!
>
> I turn off optimization and enable DEBUG_FLAGS using src.conf:
>
> CFLAGS= -O0 -pipe
> COPTFLAGS= -O0 -pipe
> DEBUG_FLAGS=-g
>
> This time however I run without any parallel make jobs and it fails with a
> different error:
>
> *** Signal 9
>
> Stop.
> make[6]: stopped in /usr/src/lib/clang/libclang
> *** Error code 1
>
> Stop.
> make[5]: stopped in /usr/src/lib/clang
> *** Error code 1
>
> Stop.
> make[4]: stopped in /usr/src/lib
> *** Error code 1
>
> Stop.
> make[3]: stopped in /usr/src
> *** Error code 1
>
> Stop.
> make[2]: stopped in /usr/src
> *** Error code 1
>
> Stop.
> make[1]: stopped in /usr/src
> *** Error code 1
>
> Stop.
> make: stopped in /usr/src
>
> Keen to hear
>
> On Sun, Aug 6, 2017 at 1:46 AM, Dimitry Andric  wrote:
>
>> On 5 Aug 2017, at 21:55, Aijaz Baig  wrote:
>> >
>> > I was a bit sceptical of this as it was failing with that same port (or
>> is clang a port by the way?) all the time. So as you suggested, I reduced
>> my '-j' number and it still fails at the very same place with the very same
>> error. Is it because the clang port doesn't allow parallel make jobs?
>> >
>> > Nonetheless, as to my query regarding building the world, based on what
>> my goals are (as I stated earlier), does it require me to actually build
>> the whole world?
>> >
>> > Regards,
>> > Aijaz Baig
>> >
>> > On Sat, Aug 5, 2017 at 2:34 PM, Dimitry Andric  wrote:
>> > On 5 Aug 2017, at 06:00, Aijaz Baig  wrote:
>> > >
>> > > I am trying to buildworld and it works well for quite some time until
>> it
>> > > tries to build the static version of the clang library where it
>> fails. The
>> > > error it spits is:
>> > >
>> > > Killed
>> > > *** [all_subdir_lib/clang/libclang] Error code 137
>> > >
>> > > make[5]: stopped in /usr/src/lib/clang
>> >
>> > I'm just guessing, but the most likely cause is lack of memory.  Please
>> > check dmesg for any messages like "pid foo (bar), uid baz, was killed:
>> > out of swap space".  To work around this, you can temporarily add more
>> > swap space, or lower your -j level.
>>
>> Hm, now I read that your obj dir is on NFS, you might be hitting some
>> 4GiB filesize limit for the final .a file.  Are you building 

Fwd: buildworld fails while building static clang library

[Aijaz Baig]

Yes you could be correct. See my reply below to Dimitry.

And if this indeed is an issue is there a workaround? Or based on my first
question, do I really need to buildworld?

-- Forwarded message --
From: Aijaz Baig <aijazba...@gmail.com>
Date: Sun, Aug 6, 2017 at 9:53 AM
Subject: Re: buildworld fails while building static clang library
To: Dimitry Andric <d...@freebsd.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>, Ed Maste <
ema...@freebsd.org>


Hello

Yes guilty as charged!!!

I turn off optimization and enable DEBUG_FLAGS using src.conf:

CFLAGS= -O0 -pipe
COPTFLAGS= -O0 -pipe
DEBUG_FLAGS=-g

This time however I run without any parallel make jobs and it fails with a
different error:

*** Signal 9

Stop.
make[6]: stopped in /usr/src/lib/clang/libclang
*** Error code 1

Stop.
make[5]: stopped in /usr/src/lib/clang
*** Error code 1

Stop.
make[4]: stopped in /usr/src/lib
*** Error code 1

Stop.
make[3]: stopped in /usr/src
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src

Keen to hear

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Aijaz Baig]

Hello

Yes guilty as charged!!!

I turn off optimization and enable DEBUG_FLAGS using src.conf:

CFLAGS= -O0 -pipe
COPTFLAGS= -O0 -pipe
DEBUG_FLAGS=-g

This time however I run without any parallel make jobs and it fails with a
different error:

*** Signal 9

Stop.
make[6]: stopped in /usr/src/lib/clang/libclang
*** Error code 1

Stop.
make[5]: stopped in /usr/src/lib/clang
*** Error code 1

Stop.
make[4]: stopped in /usr/src/lib
*** Error code 1

Stop.
make[3]: stopped in /usr/src
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src

Keen to hear

On Sun, Aug 6, 2017 at 1:46 AM, Dimitry Andric  wrote:

> On 5 Aug 2017, at 21:55, Aijaz Baig  wrote:
> >
> > I was a bit sceptical of this as it was failing with that same port (or
> is clang a port by the way?) all the time. So as you suggested, I reduced
> my '-j' number and it still fails at the very same place with the very same
> error. Is it because the clang port doesn't allow parallel make jobs?
> >
> > Nonetheless, as to my query regarding building the world, based on what
> my goals are (as I stated earlier), does it require me to actually build
> the whole world?
> >
> > Regards,
> > Aijaz Baig
> >
> > On Sat, Aug 5, 2017 at 2:34 PM, Dimitry Andric  wrote:
> > On 5 Aug 2017, at 06:00, Aijaz Baig  wrote:
> > >
> > > I am trying to buildworld and it works well for quite some time until
> it
> > > tries to build the static version of the clang library where it fails.
> The
> > > error it spits is:
> > >
> > > Killed
> > > *** [all_subdir_lib/clang/libclang] Error code 137
> > >
> > > make[5]: stopped in /usr/src/lib/clang
> >
> > I'm just guessing, but the most likely cause is lack of memory.  Please
> > check dmesg for any messages like "pid foo (bar), uid baz, was killed:
> > out of swap space".  To work around this, you can temporarily add more
> > swap space, or lower your -j level.
>
> Hm, now I read that your obj dir is on NFS, you might be hitting some
> 4GiB filesize limit for the final .a file.  Are you building world with
> a very low optimization level, and debug information on?
>
> I remember there being an issue with ar and/or ranlib choking when the
> .a files become too big.  Ed, does that ring any bells?
>
> -Dimitry
>
>


-- 

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Mark Millard]

Dimitry Andric dim at FreeBSD.org wrote on
Sat Aug 5 20:16:53 UTC 2017 :

> Hm, now I read that your obj dir is on NFS, you might be hitting some
> 4GiB filesize limit for the final .a file.  Are you building world with
> a very low optimization level, and debug information on?
> 
> I remember there being an issue with ar and/or ranlib choking when the
> .a files become too big.  Ed, does that ring any bells?

For an example of this for libWebCore.a and ar see bugzilla 206679 .

Quoting part of comment 2:


. . .
I would have guessed that the file format has some 4GB limit that might have 
been reached, explaining why the terminology reads like a file truncation 
instead of out-of-memory.

Looking around some I found one 2012 reference to code in ar that was reported 
to then look like:

  /* Catch an attempt to grow an archive past its 4Gb limit.  */
  if (archive_member_file_ptr != (file_ptr) offset)
{
  bfd_set_error (bfd_error_file_truncated);
  return FALSE;
}
. . .

===
Mark Millard
markmi at dsl-only.net

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Dimitry Andric]

On 5 Aug 2017, at 21:55, Aijaz Baig  wrote:
> 
> I was a bit sceptical of this as it was failing with that same port (or is 
> clang a port by the way?) all the time. So as you suggested, I reduced my 
> '-j' number and it still fails at the very same place with the very same 
> error. Is it because the clang port doesn't allow parallel make jobs?
> 
> Nonetheless, as to my query regarding building the world, based on what my 
> goals are (as I stated earlier), does it require me to actually build the 
> whole world?
> 
> Regards,
> Aijaz Baig
> 
> On Sat, Aug 5, 2017 at 2:34 PM, Dimitry Andric  wrote:
> On 5 Aug 2017, at 06:00, Aijaz Baig  wrote:
> >
> > I am trying to buildworld and it works well for quite some time until it
> > tries to build the static version of the clang library where it fails. The
> > error it spits is:
> >
> > Killed
> > *** [all_subdir_lib/clang/libclang] Error code 137
> >
> > make[5]: stopped in /usr/src/lib/clang
> 
> I'm just guessing, but the most likely cause is lack of memory.  Please
> check dmesg for any messages like "pid foo (bar), uid baz, was killed:
> out of swap space".  To work around this, you can temporarily add more
> swap space, or lower your -j level.

Hm, now I read that your obj dir is on NFS, you might be hitting some
4GiB filesize limit for the final .a file.  Are you building world with
a very low optimization level, and debug information on?

I remember there being an issue with ar and/or ranlib choking when the
.a files become too big.  Ed, does that ring any bells?

-Dimitry



signature.asc
Description: Message signed with OpenPGP

Re: buildworld fails while building static clang library

[Aijaz Baig]

Hi Dmitry

I was a bit sceptical of this as it was failing with that same port (or is
clang a port by the way?) all the time. So as you suggested, I reduced my
'-j' number and it still fails at the very same place with the very same
error. Is it because the clang port doesn't allow parallel make jobs?

Nonetheless, as to my query regarding building the world, based on what my
goals are (as I stated earlier), does it require me to actually build the
whole world?

Regards,
Aijaz Baig

On Sat, Aug 5, 2017 at 2:34 PM, Dimitry Andric  wrote:

> On 5 Aug 2017, at 06:00, Aijaz Baig  wrote:
> >
> > I am trying to buildworld and it works well for quite some time until it
> > tries to build the static version of the clang library where it fails.
> The
> > error it spits is:
> >
> > Killed
> > *** [all_subdir_lib/clang/libclang] Error code 137
> >
> > make[5]: stopped in /usr/src/lib/clang
>
> I'm just guessing, but the most likely cause is lack of memory.  Please
> check dmesg for any messages like "pid foo (bar), uid baz, was killed:
> out of swap space".  To work around this, you can temporarily add more
> swap space, or lower your -j level.
>
> -Dimitry
>
>


-- 

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: buildworld fails while building static clang library

[Dimitry Andric]

On 5 Aug 2017, at 06:00, Aijaz Baig  wrote:
> 
> I am trying to buildworld and it works well for quite some time until it
> tries to build the static version of the clang library where it fails. The
> error it spits is:
> 
> Killed
> *** [all_subdir_lib/clang/libclang] Error code 137
> 
> make[5]: stopped in /usr/src/lib/clang

I'm just guessing, but the most likely cause is lack of memory.  Please
check dmesg for any messages like "pid foo (bar), uid baz, was killed:
out of swap space".  To work around this, you can temporarily add more
swap space, or lower your -j level.

-Dimitry



signature.asc
Description: Message signed with OpenPGP

buildworld fails while building static clang library

[Aijaz Baig]

I am trying to buildworld and it works well for quite some time until it
tries to build the static version of the clang library where it fails. The
error it spits is:

Killed
*** [all_subdir_lib/clang/libclang] Error code 137

make[5]: stopped in /usr/src/lib/clang
1 error

make[5]: stopped in /usr/src/lib/clang
*** [all_subdir_lib/clang] Error code 2

make[4]: stopped in /usr/src/lib
1 error

make[4]: stopped in /usr/src/lib
*** [lib__L] Error code 2

make[3]: stopped in /usr/src
1 error

make[3]: stopped in /usr/src
*** [libraries] Error code 2

make[2]: stopped in /usr/src
1 error

make[2]: stopped in /usr/src
*** [_libraries] Error code 2

make[1]: stopped in /usr/src
1 error

make[1]: stopped in /usr/src
*** [buildworld] Error code 2

make: stopped in /usr/src

make: stopped in /usr/src

The number of files that are passed to the archiver are too many to list
here. Here's the pastebin link: https://pastebin.com/3ifFWVdA .

Also my MAKEOBJDIRPREFIX is pointing to an NFS share. The reason I am doing
so is that I want to create a complete sysroot so I can then use the
resulting gdb from it to debug the kernel and the KLDs on this system. That
NFS share lies on the machine which will be used to debug this one. Since
(I've read that) GDB uses a mechanism similar to libraries when debugging
loaded modules, I assume I need to first create the 'world' for this
machine on the NFS mount and then use the gdb inside to load symbols files
for the KLDs. If this is incorrect or if I am going for an overkill with
'buildworld' please sugggest an optimum way

-- 

Best Regards,
Aijaz Baig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"