Re: FYI: main (bad9fa56620e based): some unexpected SIGSEGV's using poudriere-devel to build armv7 ports on aarch64 (cortex-a72) system

2021-03-10 Thread Mark Millard via freebsd-current



On 2021-Mar-10, at 10:09, Olivier Houchard  wrote:

> On Tue, Mar 09, 2021 at 03:39:42PM -0800, Mark Millard via freebsd-arm wrote:
>> After using poudriere to build ports for native cortex-a72
>> on the MACCHIATObin Double Shot (and similarly for
>> cortex-a57 on the OverDrive 1000) I attempted to do my
>> usual bulk build targeting cortex-a7 via poudriere-devel:
>> 
>> # poudriere jail -i -jFBSDFSSDjailArmV7
>> Jail name: FBSDFSSDjailArmV7
>> Jail version:  14.0-CURRENT
>> Jail arch: arm.armv7
>> Jail method:   null
>> Jail mount:/usr/obj/DESTDIRs/clang-armv7-installworld-poud
>> Jail fs:   
>> Jail updated:  2021-01-27 14:47:10
>> Jail pkgbase:  disabled
>> 
>> But I got some SIGSEGV failures that I've never before
>> had analogous failures. I'll show the 6 backtraces.
>> They all have a similar type-of-context but in various
>> programs, summarized as (from the lldb bt outputs):
>> 
> 
> FREEBSD_COMPAT32 was indeed broken on arm64, and the process would crash
> when receiving a signal. I believe I fixed it in -CURRENT with commit
> c328f64d81079bad5064c8a387883df50ab5aaed
> 

I built and updated FreeBSD based on that vintage
and the port builds are part way through. The ports
that I had observed problems for have built just
fine and no others have failed so far.

If it all builds, it will be tomorrow sometime before
the bulk builds finish. But I figured I'd indicate
that the fix looks to have fully worked for my
context that had the problem.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: FYI: main (bad9fa56620e based): some unexpected SIGSEGV's using poudriere-devel to build armv7 ports on aarch64 (cortex-a72) system

2021-03-10 Thread Olivier Houchard
Hi Mark,

On Tue, Mar 09, 2021 at 03:39:42PM -0800, Mark Millard via freebsd-arm wrote:
> After using poudriere to build ports for native cortex-a72
> on the MACCHIATObin Double Shot (and similarly for
> cortex-a57 on the OverDrive 1000) I attempted to do my
> usual bulk build targeting cortex-a7 via poudriere-devel:
> 
> # poudriere jail -i -jFBSDFSSDjailArmV7
> Jail name: FBSDFSSDjailArmV7
> Jail version:  14.0-CURRENT
> Jail arch: arm.armv7
> Jail method:   null
> Jail mount:/usr/obj/DESTDIRs/clang-armv7-installworld-poud
> Jail fs:   
> Jail updated:  2021-01-27 14:47:10
> Jail pkgbase:  disabled
> 
> But I got some SIGSEGV failures that I've never before
> had analogous failures. I'll show the 6 backtraces.
> They all have a similar type-of-context but in various
> programs, summarized as (from the lldb bt outputs):
> 

FREEBSD_COMPAT32 was indeed broken on arm64, and the process would crash
when receiving a signal. I believe I fixed it in -CURRENT with commit
c328f64d81079bad5064c8a387883df50ab5aaed

Regards,

Olivier
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: FYI: main (bad9fa56620e based): some unexpected SIGSEGV's using poudriere-devel to build armv7 ports on aarch64 (cortex-a72) system

2021-03-09 Thread Mark Millard via freebsd-current
On 2021-Mar-9, at 17:18, Mark Millard  wrote:

> On 2021-Mar-9, at 15:39, Mark Millard  wrote:
> 
>> After using poudriere to build ports for native cortex-a72
>> on the MACCHIATObin Double Shot (and similarly for
>> cortex-a57 on the OverDrive 1000) I attempted to do my
>> usual bulk build targeting cortex-a7 via poudriere-devel:
>> 
>> # poudriere jail -i -jFBSDFSSDjailArmV7
>> Jail name: FBSDFSSDjailArmV7
>> Jail version:  14.0-CURRENT
>> Jail arch: arm.armv7
>> Jail method:   null
>> Jail mount:/usr/obj/DESTDIRs/clang-armv7-installworld-poud
>> Jail fs:   
>> Jail updated:  2021-01-27 14:47:10
>> Jail pkgbase:  disabled
>> 
>> But I got some SIGSEGV failures that I've never before
>> had analogous failures. I'll show the 6 backtraces.
>> They all have a similar type-of-context but in various
>> programs, summarized as (from the lldb bt outputs):
>> 
>> gmake`new_job(file=) [3 examples]
>> and:
>> sh`waitcmdloop(job=0x00064230) [2 examples]
>> and:
>> cmake`(anonymous namespace)::RunCommand(command=, 
>> output="14.0-CURRENT\n", retVal=, dir=, 
>> verbose=, encoding=Auto)
>> 
>> (Only 83 ports built of 208 built, 5 failed, and 120 were
>> skipped.)
>> 
>> I have not yet tried simply running poudriere again to see
>> how reliable the specific failures may or may not be: I'm
>> first collecting and reporting this information. Nor have
>> I tried doing the build on the cortex-a57 context instead.
> 
> I have a little re-run information now:
> 
> No -JN with ALLOW_MAKE_JOBS=yes: seems repeatable
> -J1 with ALLOW_MAKE_JOBS=yes: seems repeatable
> -J1 without ALLOW_MAKE_JOBS: mixed so far in the one run.
> 
> For -J1 without ALLOW_MAKE_JOBS for as far as the bulk
> build has gotten (still in progress):
> 
> [00:22:03] [01] [00:21:12] Finished devel/cmake | cmake-3.19.6: Failed: 
> configure
> [00:23:04] [01] [00:00:59] Finished textproc/libxslt | libxslt-1.1.34_1: 
> Success
> [00:33:35] [01] [00:00:19] Finished textproc/itstool | itstool-2.0.6: Failed: 
> configure
> 
> Some ports dependent on libxslt-1.1.34_1 are building as well,
> devel/glib20 , textproc/minixmlto , and x11/xkeyboard-config
> have built.
> 
> I'm not sure it is reasonable to wait on -J1 without
> ALLOW_MAKE_JOBS to see what the devel/gdb and
> x11-toolkits/libXaw build does. For example qt5-core
> that is now building and may not be worth waiting for.
> 
> The results do suggest that the issue is racy, no matter
> if the number of active processes/threads changes the
> probabilities involved or not.

I stopped the build and continued by starting
another . . .

Without both -J and ALLOW_MAKE_JOBS ((so implicitly
-J4 in the context) it got the following results
for the 4-left of 5 originally failing:

[00:00:51] [02] [00:00:19] Finished textproc/itstool | itstool-2.0.6: Failed: 
configure
[00:05:00] [02] [00:02:38] Finished x11-toolkits/libXaw | libXaw-1.0.13_3,2: 
Success
[00:19:02] [03] [00:15:00] Finished devel/gdb@py37 | gdb-10.1_1: Failed: build
[00:29:33] [01] [00:29:01] Finished devel/cmake | cmake-3.19.6: Failed: 
configure

17 ports built that depend on textproc/libxslt and/or
x11-toolkits/libXaw . The 3 failures above made the
rest (100) skip.

textproc/libxslt and x11-toolkits/libXaw sometimes failing
and sometimes not suggests a racy context at some level
of the operations involved.



>> I'll note that when I looked at detail as the assembler level
>> it appeared that there was a frame not shown between #0 and #1
>> in lldb's output: Frame #1 "->" was indicating the instruction
>> after a simple bl to a not-shown subroutine.
>> 
>> For building textproc/libxslt :
>> (jobserver_acquire not shown between #0 and #1)
>> 
>> (lldb) bt
>> * thread #1, name = 'gmake', stop reason = signal SIGSEGV
>> * frame #0: 0xe190
>>   frame #1: 0x0003b5f8 gmake`new_job(file=) at job.c:1870:21
>>   frame #2: 0x0002db80 gmake`execute_file_commands(file=) at 
>> commands.c:476:3 [artificial]
>>   frame #3: 0x00049acc gmake`update_file [inlined] 
>> remake_file(file=0x400a9700) at remake.c:1234:11
>>   frame #4: 0x00049a84 gmake`update_file [inlined] 
>> update_file_1(file=, depth=6) at remake.c:835
>>   frame #5: 0x000494ec gmake`update_file(file=, 
>> depth=) at remake.c:336
>>   frame #6: 0x0004b08c gmake`check_dep(file=0x400a9700, depth=, 
>> this_mtime=1, must_make_ptr=0xc4ac) at remake.c:1024:20
>>   frame #7: 0x00049074 gmake`update_file at remake.c:572:17
>>   frame #8: 0x00048b80 gmake`update_file(file=, 
>> depth=) at remake.c:336
>>   frame #9: 0x0004b08c gmake`check_dep(file=0x400a9400, depth=, 
>> this_mtime=1, must_make_ptr=0xc564) at remake.c:1024:20
>>   frame #10: 0x00049074 gmake`update_file at remake.c:572:17
>>   frame #11: 0x00048b80 gmake`update_file(file=, 
>> depth=) at remake.c:336
>>   frame #12: 0x0004b08c gmake`check_dep(file=0x400a8f20, 
>> depth=, this_mtime=1, must_make_ptr=0xc61c) at 
>> remake.c:1024:20
>>   frame #13: 0x00049074 gmake`update_file at rema

Re: FYI: main (bad9fa56620e based): some unexpected SIGSEGV's using poudriere-devel to build armv7 ports on aarch64 (cortex-a72) system

2021-03-09 Thread Mark Millard via freebsd-current
On 2021-Mar-9, at 15:39, Mark Millard  wrote:

> After using poudriere to build ports for native cortex-a72
> on the MACCHIATObin Double Shot (and similarly for
> cortex-a57 on the OverDrive 1000) I attempted to do my
> usual bulk build targeting cortex-a7 via poudriere-devel:
> 
> # poudriere jail -i -jFBSDFSSDjailArmV7
> Jail name: FBSDFSSDjailArmV7
> Jail version:  14.0-CURRENT
> Jail arch: arm.armv7
> Jail method:   null
> Jail mount:/usr/obj/DESTDIRs/clang-armv7-installworld-poud
> Jail fs:   
> Jail updated:  2021-01-27 14:47:10
> Jail pkgbase:  disabled
> 
> But I got some SIGSEGV failures that I've never before
> had analogous failures. I'll show the 6 backtraces.
> They all have a similar type-of-context but in various
> programs, summarized as (from the lldb bt outputs):
> 
> gmake`new_job(file=) [3 examples]
> and:
> sh`waitcmdloop(job=0x00064230) [2 examples]
> and:
> cmake`(anonymous namespace)::RunCommand(command=, 
> output="14.0-CURRENT\n", retVal=, dir=, 
> verbose=, encoding=Auto)
> 
> (Only 83 ports built of 208 built, 5 failed, and 120 were
> skipped.)
> 
> I have not yet tried simply running poudriere again to see
> how reliable the specific failures may or may not be: I'm
> first collecting and reporting this information. Nor have
> I tried doing the build on the cortex-a57 context instead.

I have a little re-run information now:

No -JN with ALLOW_MAKE_JOBS=yes: seems repeatable
-J1 with ALLOW_MAKE_JOBS=yes: seems repeatable
-J1 without ALLOW_MAKE_JOBS: mixed so far in the one run.

For -J1 without ALLOW_MAKE_JOBS for as far as the bulk
build has gotten (still in progress):

[00:22:03] [01] [00:21:12] Finished devel/cmake | cmake-3.19.6: Failed: 
configure
[00:23:04] [01] [00:00:59] Finished textproc/libxslt | libxslt-1.1.34_1: Success
[00:33:35] [01] [00:00:19] Finished textproc/itstool | itstool-2.0.6: Failed: 
configure

Some ports dependent on libxslt-1.1.34_1 are building as well,
devel/glib20 , textproc/minixmlto , and x11/xkeyboard-config
have built.

I'm not sure it is reasonable to wait on -J1 without
ALLOW_MAKE_JOBS to see what the devel/gdb and
x11-toolkits/libXaw build does. For example qt5-core
that is now building and may not be worth waiting for.

The results do suggest that the issue is racy, no matter
if the number of active processes/threads changes the
probabilities involved or not.

> I'll note that when I looked at detail as the assembler level
> it appeared that there was a frame not shown between #0 and #1
> in lldb's output: Frame #1 "->" was indicating the instruction
> after a simple bl to a not-shown subroutine.
> 
> For building textproc/libxslt :
> (jobserver_acquire not shown between #0 and #1)
> 
> (lldb) bt
> * thread #1, name = 'gmake', stop reason = signal SIGSEGV
>  * frame #0: 0xe190
>frame #1: 0x0003b5f8 gmake`new_job(file=) at job.c:1870:21
>frame #2: 0x0002db80 gmake`execute_file_commands(file=) at 
> commands.c:476:3 [artificial]
>frame #3: 0x00049acc gmake`update_file [inlined] 
> remake_file(file=0x400a9700) at remake.c:1234:11
>frame #4: 0x00049a84 gmake`update_file [inlined] 
> update_file_1(file=, depth=6) at remake.c:835
>frame #5: 0x000494ec gmake`update_file(file=, 
> depth=) at remake.c:336
>frame #6: 0x0004b08c gmake`check_dep(file=0x400a9700, depth=, 
> this_mtime=1, must_make_ptr=0xc4ac) at remake.c:1024:20
>frame #7: 0x00049074 gmake`update_file at remake.c:572:17
>frame #8: 0x00048b80 gmake`update_file(file=, 
> depth=) at remake.c:336
>frame #9: 0x0004b08c gmake`check_dep(file=0x400a9400, depth=, 
> this_mtime=1, must_make_ptr=0xc564) at remake.c:1024:20
>frame #10: 0x00049074 gmake`update_file at remake.c:572:17
>frame #11: 0x00048b80 gmake`update_file(file=, 
> depth=) at remake.c:336
>frame #12: 0x0004b08c gmake`check_dep(file=0x400a8f20, 
> depth=, this_mtime=1, must_make_ptr=0xc61c) at 
> remake.c:1024:20
>frame #13: 0x00049074 gmake`update_file at remake.c:572:17
>frame #14: 0x00048b80 gmake`update_file(file=, 
> depth=) at remake.c:336
>frame #15: 0x000487e0 gmake`update_goal_chain(goaldeps=) at 
> remake.c:151:22
>frame #16: 0x0003f25c gmake`main(argc=2, argv=0xd470, envp=0x) 
> at main.c:2589:13
>frame #17: 0x0002c0fc gmake`__start(argc=2, argv=, 
> env=, ps_strings=, obj=0x400c4004, 
> cleanup=0x40091aa0) at crt1_c.c:92:7
> 
> -> 1870   got_token = jobserver_acquire (waiting_jobs != NULL);
> 
>0x3b5f4 <+1288>: bl 0x50078   ; jobserver_acquire at 
> posixos.c:265
> ->  0x3b5f8 <+1292>: cmpr0, #1
> 
> 
> For building x11-toolkits/libXaw :
> (jobserver_acquire not shown between #0 and #1)
> 
> (lldb) bt
> * thread #1, name = 'gmake', stop reason = signal SIGSEGV
>  * frame #0: 0xe190
>frame #1: 0x0003b5f8 gmake`new_job(file=) at job.c:1870:21
>frame #2: 0x0002db80 gmake`execute_file_commands(file=) at 
> commands.c:476:3 [artific

FYI: main (bad9fa56620e based): some unexpected SIGSEGV's using poudriere-devel to build armv7 ports on aarch64 (cortex-a72) system

2021-03-09 Thread Mark Millard via freebsd-current
After using poudriere to build ports for native cortex-a72
on the MACCHIATObin Double Shot (and similarly for
cortex-a57 on the OverDrive 1000) I attempted to do my
usual bulk build targeting cortex-a7 via poudriere-devel:

# poudriere jail -i -jFBSDFSSDjailArmV7
Jail name: FBSDFSSDjailArmV7
Jail version:  14.0-CURRENT
Jail arch: arm.armv7
Jail method:   null
Jail mount:/usr/obj/DESTDIRs/clang-armv7-installworld-poud
Jail fs:   
Jail updated:  2021-01-27 14:47:10
Jail pkgbase:  disabled

But I got some SIGSEGV failures that I've never before
had analogous failures. I'll show the 6 backtraces.
They all have a similar type-of-context but in various
programs, summarized as (from the lldb bt outputs):

gmake`new_job(file=) [3 examples]
and:
sh`waitcmdloop(job=0x00064230) [2 examples]
and:
cmake`(anonymous namespace)::RunCommand(command=, 
output="14.0-CURRENT\n", retVal=, dir=, 
verbose=, encoding=Auto)

(Only 83 ports built of 208 built, 5 failed, and 120 were
skipped.)

I have not yet tried simply running poudriere again to see
how reliable the specific failures may or may not be: I'm
first collecting and reporting this information. Nor have
I tried doing the build on the cortex-a57 context instead.



I'll note that when I looked at detail as the assembler level
it appeared that there was a frame not shown between #0 and #1
in lldb's output: Frame #1 "->" was indicating the instruction
after a simple bl to a not-shown subroutine.

For building textproc/libxslt :
(jobserver_acquire not shown between #0 and #1)

(lldb) bt
* thread #1, name = 'gmake', stop reason = signal SIGSEGV
  * frame #0: 0xe190
frame #1: 0x0003b5f8 gmake`new_job(file=) at job.c:1870:21
frame #2: 0x0002db80 gmake`execute_file_commands(file=) at 
commands.c:476:3 [artificial]
frame #3: 0x00049acc gmake`update_file [inlined] 
remake_file(file=0x400a9700) at remake.c:1234:11
frame #4: 0x00049a84 gmake`update_file [inlined] 
update_file_1(file=, depth=6) at remake.c:835
frame #5: 0x000494ec gmake`update_file(file=, 
depth=) at remake.c:336
frame #6: 0x0004b08c gmake`check_dep(file=0x400a9700, depth=, 
this_mtime=1, must_make_ptr=0xc4ac) at remake.c:1024:20
frame #7: 0x00049074 gmake`update_file at remake.c:572:17
frame #8: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #9: 0x0004b08c gmake`check_dep(file=0x400a9400, depth=, 
this_mtime=1, must_make_ptr=0xc564) at remake.c:1024:20
frame #10: 0x00049074 gmake`update_file at remake.c:572:17
frame #11: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #12: 0x0004b08c gmake`check_dep(file=0x400a8f20, depth=, 
this_mtime=1, must_make_ptr=0xc61c) at remake.c:1024:20
frame #13: 0x00049074 gmake`update_file at remake.c:572:17
frame #14: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #15: 0x000487e0 gmake`update_goal_chain(goaldeps=) at 
remake.c:151:22
frame #16: 0x0003f25c gmake`main(argc=2, argv=0xd470, envp=0x) 
at main.c:2589:13
frame #17: 0x0002c0fc gmake`__start(argc=2, argv=, 
env=, ps_strings=, obj=0x400c4004, 
cleanup=0x40091aa0) at crt1_c.c:92:7

-> 1870 got_token = jobserver_acquire (waiting_jobs != NULL);

0x3b5f4 <+1288>: bl 0x50078   ; jobserver_acquire at 
posixos.c:265
->  0x3b5f8 <+1292>: cmpr0, #1


For building x11-toolkits/libXaw :
(jobserver_acquire not shown between #0 and #1)

(lldb) bt
* thread #1, name = 'gmake', stop reason = signal SIGSEGV
  * frame #0: 0xe190
frame #1: 0x0003b5f8 gmake`new_job(file=) at job.c:1870:21
frame #2: 0x0002db80 gmake`execute_file_commands(file=) at 
commands.c:476:3 [artificial]
frame #3: 0x00049acc gmake`update_file [inlined] 
remake_file(file=0x4036a580) at remake.c:1234:11
frame #4: 0x00049a84 gmake`update_file [inlined] 
update_file_1(file=, depth=6) at remake.c:835
frame #5: 0x000494ec gmake`update_file(file=, 
depth=) at remake.c:336
frame #6: 0x0004b08c gmake`check_dep(file=0x4036a580, depth=, 
this_mtime=1, must_make_ptr=0xc31c) at remake.c:1024:20
frame #7: 0x00049074 gmake`update_file at remake.c:572:17
frame #8: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #9: 0x0004b08c gmake`check_dep(file=0x4036a220, depth=, 
this_mtime=1, must_make_ptr=0xc3d4) at remake.c:1024:20
frame #10: 0x00049074 gmake`update_file at remake.c:572:17
frame #11: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #12: 0x0004b08c gmake`check_dep(file=0x40369ec0, depth=, 
this_mtime=1, must_make_ptr=0xc48c) at remake.c:1024:20
frame #13: 0x00049074 gmake`update_file at remake.c:572:17
frame #14: 0x00048b80 gmake`update_file(file=, 
depth=) at remake.c:336
frame #15: 0x000487e0 gmake`update_goal_chain(goaldeps=) at 
remake.c:151:22
frame #16: 0x0003f25c gmake`main(argc=2, argv=0xd500, envp=0x) 
at main.c:25