subject:"\[OE\-core\] bitbake\-server does not exit after build ends"

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-08 Thread Zoltan Boszormenyi


2024. 01. 05. 17:54 keltezéssel, Martin Jansa írta:
On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org 
  wrote:


FWIW: I see the processes running after bitbake exit only in kirkstone and 
only with
PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together 
with
bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin   4184043  0.0  0.0   9124  2048 pts/4    S+ 16:47   0:00 grep 
--colour=auto
luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile: 
/OE/build/luneos-kirkstone/cache/prserv.sqlite3,
Address: 127.0.0.1:35811 , PID: 4184924
Loading cache: 100%

|##|
Time: 0:00:04
Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% 
match, 100%
complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun 
and all
succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin     51324  0.0  0.0   9124  2048 pts/4    S+ 16:54   0:00 grep 
--colour=auto
luneos-kirkstone
martin   4184584  8.9  0.4 668196 536892 ?       Sl 16:47   0:37 
bitbake-server
/OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
/OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock 
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184924  0.0  0.1 340280 196428 ?       S  16:47   0:00 
bitbake-server
/OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
/OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock 
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184927  0.0  0.1 340328 194440 ?       S  16:47   0:00 
bitbake-server
/OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
/OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock 
/OE/build/luneos-kirkstone/bitbake.sock 0
None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 
2024-01-05
16:47:38.367526 ---
4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 
'force':
False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [],
'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': 
[],
'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 
'skipsetscene':
False, 'runall': None, 'runonly': None, 'writeeventlog': None,
'build_verbose_shell': False, 'build_verbose_stdout': False, 
'default_loglevel': 20,
'debug_domains': {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS':
'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy 
ALL_PROXY
no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE 
BB_NUMBER_THREADS
PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
'SSH_AUTH_SOCK': '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID': 
'1894', 'PWD':
'/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': 
'/home/martin-gentoo',
'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':

'/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER':
'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 'WINDOWID':
'35651587', 'COLORTERM': 'truecolor', 'CSF_MDTVTexturesDirectory':
'/usr/share/opencascade/resources/Textures', 'MACHINES': 'qemux86-64',
'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M --shift 5', 
'XDG_SESSION_PATH':
'/org/freedesktop/DisplayManager/Session1', 'XDG_MENU_PREFIX': 'xfce-', 
'JDK_HOME':

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-05 Thread Steve Sakoman

On Fri, Jan 5, 2024 at 7:02 AM Steve Sakoman via
lists.openembedded.org 
wrote:
>
> On Fri, Jan 5, 2024 at 6:54 AM Martin Jansa  wrote:
> >
> > On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org 
> >  wrote:
> >>
> >> FWIW: I see the processes running after bitbake exit only in kirkstone and 
> >> only with
> >> PRSERV_HOST = "localhost:0"
> >> if I drop PRSERV_HOST then all 3 bitbake-server processes exist together 
> >> with bitbake itself.
> >>
> >> luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
> >> luneos-kirkstone $ ps aux | grep luneos-kirkstone
> >> martin   4184043  0.0  0.0   9124  2048 pts/4S+   16:47   0:00 grep 
> >> --colour=auto luneos-kirkstone
> >> luneos-kirkstone $ bitbake -k zlib-native
> >> NOTE: Started PRServer with DBfile: 
> >> /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811, 
> >> PID: 4184924
> >> Loading cache: 100% 
> >> |##|
> >>  Time: 0:00:04
> >> Loaded 4570 entries from dependency cache.
> >> ...
> >> Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% 
> >> match, 100% complete)
> >> NOTE: Executing Tasks
> >> NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be 
> >> rerun and all succeeded.
> >> NOTE: Writing buildhistory
> >> NOTE: Writing buildhistory took: 15 seconds
> >> luneos-kirkstone $ ps aux | grep luneos-kirkstone
> >> martin 51324  0.0  0.0   9124  2048 pts/4S+   16:54   0:00 grep 
> >> --colour=auto luneos-kirkstone
> >> martin   4184584  8.9  0.4 668196 536892 ?   Sl   16:47   0:37 
> >> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
> >> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
> >> /OE/build/luneos-kirkstone/bitbake.lock 
> >> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
> >> martin   4184924  0.0  0.1 340280 196428 ?   S16:47   0:00 
> >> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
> >> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
> >> /OE/build/luneos-kirkstone/bitbake.lock 
> >> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
> >> martin   4184927  0.0  0.1 340328 194440 ?   S16:47   0:00 
> >> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
> >> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
> >> /OE/build/luneos-kirkstone/bitbake.lock 
> >> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
> >>
> >> luneos-kirkstone $ cat bitbake-cookerdaemon.log
> >> 4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 
> >> 2024-01-05 16:47:38.367526 ---
> >> 4184584 16:47:38.370315 Started bitbake server pid 4184584
> >> 4184584 16:47:38.370831 Entering server connection loop
> >> 4184584 16:47:38.372043 Accepting [ >> proto=0, laddr=bitbake.sock>] ([])
> >> 4184584 16:47:38.376431 Processing Client
> >> 4184584 16:47:38.376633 Connecting Client
> >> 4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
> >> 4184584 16:47:38.379355 Command Completed
> >> 4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 
> >> 'force': False, 'invalidate_stamp': None, 'dry_run': False, 
> >> 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 
> >> 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': 
> >> False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 
> >> 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 
> >> 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': 
> >> {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE 
> >> DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY 
> >> no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE 
> >> BB_NUMBER_THREADS PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE 
> >> SOCKS5_PASSWD SOCKS5_USER WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED 
> >> PSEUDO_BUILD', 'DISTRO': 'luneos', 'SSH_AUTH_SOCK': 
> >> '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID': '1894', 'PWD': 
> >> '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': 
> >> '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH': 
> >> '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
> >>  'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 
> >> 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 
> >> 'WINDOWID': '35651587', 'COLORTERM': 'truecolor', 
> >> 'CSF_MDTVTexturesDirectory':

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-05 Thread Joshua Watt

On Fri, Jan 5, 2024, 9:54 AM Martin Jansa  wrote:

> On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org
>  wrote:
>
>> FWIW: I see the processes running after bitbake exit only in kirkstone
>> and only with
>> PRSERV_HOST = "localhost:0"
>> if I drop PRSERV_HOST then all 3 bitbake-server processes exist together
>> with bitbake itself.
>>
>> luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
>> luneos-kirkstone $ ps aux | grep luneos-kirkstone
>> martin   4184043  0.0  0.0   9124  2048 pts/4S+   16:47   0:00 grep
>> --colour=auto luneos-kirkstone
>> luneos-kirkstone $ bitbake -k zlib-native
>> NOTE: Started PRServer with DBfile:
>> /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
>> PID: 4184924
>> Loading cache: 100%
>> |##|
>> Time: 0:00:04
>> Loaded 4570 entries from dependency cache.
>> ...
>> Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
>> match, 100% complete)
>> NOTE: Executing Tasks
>> NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be
>> rerun and all succeeded.
>> NOTE: Writing buildhistory
>> NOTE: Writing buildhistory took: 15 seconds
>> luneos-kirkstone $ ps aux | grep luneos-kirkstone
>> martin 51324  0.0  0.0   9124  2048 pts/4S+   16:54   0:00 grep
>> --colour=auto luneos-kirkstone
>> martin   4184584  8.9  0.4 668196 536892 ?   Sl   16:47   0:37
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
>> /OE/build/luneos-kirkstone/bitbake.lock
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>> martin   4184924  0.0  0.1 340280 196428 ?   S16:47   0:00
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
>> /OE/build/luneos-kirkstone/bitbake.lock
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>> martin   4184927  0.0  0.1 340328 194440 ?   S16:47   0:00
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
>> /OE/build/luneos-kirkstone/bitbake.lock
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>>
>> luneos-kirkstone $ cat bitbake-cookerdaemon.log
>> 4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at
>> 2024-01-05 16:47:38.367526 ---
>> 4184584 16:47:38.370315 Started bitbake server pid 4184584
>> 4184584 16:47:38.370831 Entering server connection loop
>> 4184584 16:47:38.372043 Accepting [> proto=0, laddr=bitbake.sock>] ([])
>> 4184584 16:47:38.376431 Processing Client
>> 4184584 16:47:38.376633 Connecting Client
>> 4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
>> 4184584 16:47:38.379355 Command Completed
>> 4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
>> 'force': False, 'invalidate_stamp': None, 'dry_run': False,
>> 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
>> 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
>> 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
>> None, 'writeeventlog': None, 'build_verbose_shell': False,
>> 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
>> {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
>> TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
>> SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
>> PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
>> WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
>> 'SSH_AUTH_SOCK': '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
>> '1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
>> '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
>> '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
>> 'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 
>> 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
>> 'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
>> 'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
>> 'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
>> --shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
>> 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
>> '/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
>> '/etc/sandbox.d

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-05 Thread Steve Sakoman

On Fri, Jan 5, 2024 at 6:54 AM Martin Jansa  wrote:
>
> On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org 
>  wrote:
>>
>> FWIW: I see the processes running after bitbake exit only in kirkstone and 
>> only with
>> PRSERV_HOST = "localhost:0"
>> if I drop PRSERV_HOST then all 3 bitbake-server processes exist together 
>> with bitbake itself.
>>
>> luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
>> luneos-kirkstone $ ps aux | grep luneos-kirkstone
>> martin   4184043  0.0  0.0   9124  2048 pts/4S+   16:47   0:00 grep 
>> --colour=auto luneos-kirkstone
>> luneos-kirkstone $ bitbake -k zlib-native
>> NOTE: Started PRServer with DBfile: 
>> /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811, 
>> PID: 4184924
>> Loading cache: 100% 
>> |##|
>>  Time: 0:00:04
>> Loaded 4570 entries from dependency cache.
>> ...
>> Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% 
>> match, 100% complete)
>> NOTE: Executing Tasks
>> NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun 
>> and all succeeded.
>> NOTE: Writing buildhistory
>> NOTE: Writing buildhistory took: 15 seconds
>> luneos-kirkstone $ ps aux | grep luneos-kirkstone
>> martin 51324  0.0  0.0   9124  2048 pts/4S+   16:54   0:00 grep 
>> --colour=auto luneos-kirkstone
>> martin   4184584  8.9  0.4 668196 536892 ?   Sl   16:47   0:37 
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
>> /OE/build/luneos-kirkstone/bitbake.lock 
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>> martin   4184924  0.0  0.1 340280 196428 ?   S16:47   0:00 
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
>> /OE/build/luneos-kirkstone/bitbake.lock 
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>> martin   4184927  0.0  0.1 340328 194440 ?   S16:47   0:00 
>> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server 
>> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log 
>> /OE/build/luneos-kirkstone/bitbake.lock 
>> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>>
>> luneos-kirkstone $ cat bitbake-cookerdaemon.log
>> 4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 
>> 2024-01-05 16:47:38.367526 ---
>> 4184584 16:47:38.370315 Started bitbake server pid 4184584
>> 4184584 16:47:38.370831 Entering server connection loop
>> 4184584 16:47:38.372043 Accepting [> proto=0, laddr=bitbake.sock>] ([])
>> 4184584 16:47:38.376431 Processing Client
>> 4184584 16:47:38.376633 Connecting Client
>> 4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
>> 4184584 16:47:38.379355 Command Completed
>> 4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 
>> 'force': False, 'invalidate_stamp': None, 'dry_run': False, 
>> 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 
>> 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 
>> 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': 
>> None, 'writeeventlog': None, 'build_verbose_shell': False, 
>> 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, 
>> {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO 
>> TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy 
>> SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS 
>> PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER 
>> WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos', 
>> 'SSH_AUTH_SOCK': '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID': 
>> '1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': 
>> '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH': 
>> '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
>>  'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 
>> 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 'WINDOWID': 
>> '35651587', 'COLORTERM': 'truecolor', 'CSF_MDTVTexturesDirectory': 
>> '/usr/share/opencascade/resources/Textures', 'MACHINES': 'qemux86-64', 
>> 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M --shift 5', 
>> 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1', 
>> 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME': 
>> '/etc/java-config-2/current-system-vm',

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-05 Thread Martin Jansa

On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org
 wrote:

> FWIW: I see the processes running after bitbake exit only in kirkstone and
> only with
> PRSERV_HOST = "localhost:0"
> if I drop PRSERV_HOST then all 3 bitbake-server processes exist together
> with bitbake itself.
>
> luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
> luneos-kirkstone $ ps aux | grep luneos-kirkstone
> martin   4184043  0.0  0.0   9124  2048 pts/4S+   16:47   0:00 grep
> --colour=auto luneos-kirkstone
> luneos-kirkstone $ bitbake -k zlib-native
> NOTE: Started PRServer with DBfile:
> /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
> PID: 4184924
> Loading cache: 100%
> |##|
> Time: 0:00:04
> Loaded 4570 entries from dependency cache.
> ...
> Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
> match, 100% complete)
> NOTE: Executing Tasks
> NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be
> rerun and all succeeded.
> NOTE: Writing buildhistory
> NOTE: Writing buildhistory took: 15 seconds
> luneos-kirkstone $ ps aux | grep luneos-kirkstone
> martin 51324  0.0  0.0   9124  2048 pts/4S+   16:54   0:00 grep
> --colour=auto luneos-kirkstone
> martin   4184584  8.9  0.4 668196 536892 ?   Sl   16:47   0:37
> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
> /OE/build/luneos-kirkstone/bitbake.lock
> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
> martin   4184924  0.0  0.1 340280 196428 ?   S16:47   0:00
> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
> /OE/build/luneos-kirkstone/bitbake.lock
> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
> martin   4184927  0.0  0.1 340328 194440 ?   S16:47   0:00
> bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
> decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
> /OE/build/luneos-kirkstone/bitbake.lock
> /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
>
> luneos-kirkstone $ cat bitbake-cookerdaemon.log
> 4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at
> 2024-01-05 16:47:38.367526 ---
> 4184584 16:47:38.370315 Started bitbake server pid 4184584
> 4184584 16:47:38.370831 Entering server connection loop
> 4184584 16:47:38.372043 Accepting [ proto=0, laddr=bitbake.sock>] ([])
> 4184584 16:47:38.376431 Processing Client
> 4184584 16:47:38.376633 Connecting Client
> 4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
> 4184584 16:47:38.379355 Command Completed
> 4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
> 'force': False, 'invalidate_stamp': None, 'dry_run': False,
> 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
> 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
> 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
> None, 'writeeventlog': None, 'build_verbose_shell': False,
> 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
> {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
> TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
> SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
> PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
> WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
> 'SSH_AUTH_SOCK': '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
> '1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
> '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
> '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
> 'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 
> 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
> 'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
> 'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
> 'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
> --shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
> 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
> '/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
> '/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf
> /etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP':
> 'SC|screen.xterm-256color|VT

Re: [OE-core] bitbake-server does not exit after build ends

2024-01-05 Thread Martin Jansa

FWIW: I see the processes running after bitbake exit only in kirkstone and
only with
PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together
with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin   4184043  0.0  0.0   9124  2048 pts/4S+   16:47   0:00 grep
--colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile:
/OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
PID: 4184924
Loading cache: 100%
|##|
Time: 0:00:04
Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun
and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 51324  0.0  0.0   9124  2048 pts/4S+   16:54   0:00 grep
--colour=auto luneos-kirkstone
martin   4184584  8.9  0.4 668196 536892 ?   Sl   16:47   0:37
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184924  0.0  0.1 340280 196428 ?   S16:47   0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184927  0.0  0.1 340328 194440 ?   S16:47   0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at
2024-01-05 16:47:38.367526 ---
4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
'force': False, 'invalidate_stamp': None, 'dry_run': False,
'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
None, 'writeeventlog': None, 'build_verbose_shell': False,
'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
{}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
'SSH_AUTH_SOCK': '/tmp/ssh-XX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
'1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
'/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
'/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER':
'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
--shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
'/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
'/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf
/etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP':
'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual

Re: [OE-core] bitbake-server does not exit after build ends

2023-12-31 Thread Alexander Kanavin

For what it's worth, I am not seeing any of these issues (hanging
processes after bitbake exit, timeouts on world builds with all
meta-oe-layers) with python 3.12 provided by buildtools-tarball. The
issues are probably real, but it's also possible python 3.12 per se is
not to blame.

It would probably help if you check if what you're seeing occurs with
python 3.12 from the tarball, which you can build from this branch:
https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/py3.12
(the log in the web ui isn't updated yet, I just pushed)

Alex

On Fri, 22 Dec 2023 at 00:06, Randy MacLeod via lists.openembedded.org
 wrote:
>
> On 2023-12-21 1:56 p.m., Martin Jansa wrote:
>
> On Thu, Dec 21, 2023 at 7:47 PM Randy MacLeod  
> wrote:
>>
>> On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org wrote:
>>
>> So it's not just Fedora, but Python 3.12 in general.
>>
>> Googling for problem with threads + fork reveals that
>> it is indeed an issue even in C / POSIX: only the thread
>> executing fork() will become part of the new process,
>> but not the main process starting the thread or any
>> other threads from the original process.
>>
>> The warning is justified, and apparently the issue
>> not occurring in bitbake with previous Python versions
>> is pure luck.
>>
>> FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.
>
>
> But this fixes it only for bitbake master, right?
>
> Even master might still have some problems.
> I can do a simple build of a package but in doing:
>
> $ bitbake -k world
>
> I see:
> NOTE: No reply from server in 30s (for command ping at 17:50:35.510798)
>
> After some time.
>
> I'm not going to try to debug it at this point in the day/year!
>
>
> With bitbake 2.0 used in kirkstone I still see bitbake processes still 
> running after the build is finished.
>
> I didn't try kirkstone. Maybe Steve has that on his list of things to do in 
> his spare time! ;-)
>
> ../Randy
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193086): 
https://lists.openembedded.org/g/openembedded-core/message/193086
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-12-21 Thread Randy MacLeod via lists.openembedded.org


On 2023-12-21 1:56 p.m., Martin Jansa wrote:
On Thu, Dec 21, 2023 at 7:47 PM Randy MacLeod 
 wrote:


On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via
lists.openembedded.org


wrote:

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that
it is indeed an issue even in C / POSIX: only the thread
executing fork() will become part of the new process,
but not the main process starting the thread or any
other threads from the original process.

The warning is justified, and apparently the issue
not occurring in bitbake with previous Python versions
is pure luck.


FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.


But this fixes it only for bitbake master, right?

Even master might still have some problems.
I can do a simple build of a package but in doing:

$ bitbake -k world

I see:
NOTE: No reply from server in 30s (for command ping at 17:50:35.510798)

After some time.

I'm not going to try to debug it at this point in the day/year!



With bitbake 2.0 used in kirkstone I still see bitbake processes still 
running after the build is finished.


I didn't try kirkstone. Maybe Steve has that on his list of things to do 
in his spare time! ;-)


../Randy


--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192854): 
https://lists.openembedded.org/g/openembedded-core/message/192854
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-12-21 Thread Martin Jansa

On Thu, Dec 21, 2023 at 7:47 PM Randy MacLeod 
wrote:

> On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org
> wrote:
>
> So it's not just Fedora, but Python 3.12 in general.
>
> Googling for problem with threads + fork reveals that
> it is indeed an issue even in C / POSIX: only the thread
> executing fork() will become part of the new process,
> but not the main process starting the thread or any
> other threads from the original process.
>
> The warning is justified, and apparently the issue
> not occurring in bitbake with previous Python versions
> is pure luck.
>
> FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.
>

But this fixes it only for bitbake master, right?

With bitbake 2.0 used in kirkstone I still see bitbake processes still
running after the build is finished.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192845): 
https://lists.openembedded.org/g/openembedded-core/message/192845
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-12-21 Thread Randy MacLeod via lists.openembedded.org

On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org 
wrote:

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that
it is indeed an issue even in C / POSIX: only the thread
executing fork() will become part of the new process,
but not the main process starting the thread or any
other threads from the original process.

The warning is justified, and apparently the issue
not occurring in bitbake with previous Python versions
is pure luck.


FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.


--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192844): 
https://lists.openembedded.org/g/openembedded-core/message/192844
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-29 Thread Zoltan Boszormenyi


2023. 11. 28. 15:50 keltezéssel, Alexander Kanavin írta:

On Tue, 28 Nov 2023 at 11:31, Zoltan Boszormenyi  wrote:

The warning only occurs if os.fork() is called when
there are already Python threads.

I guess the general performance uplift in Python 3.12
can trigger a pre-existing issue in bitbake more easily.

It is entirely possible that the threads bitbake creates do not
exist yet with Python 3.11 and earlier at the time of executing
os.fork(). ¯\_(ツ)_/¯

I think you might be too quick to jump to conclusions here.


Indeed.
I should have added "assuming Python is not the buggy party here".
Sorry.


  I've fixed
the async issue I've been observing (backporting
https://github.com/python/cpython/commit/2e5d4e24ebcd671915427b6fd35a5b0eea412c4a
), and when running bitbake with 3.12 from buildtools-tarball,
everything works fine, and I'm not seeing any leftover processes. Poky
revision is current master:
https://git.yoctoproject.org/poky/commit/?h=master=3e50e45917831d9da2d86e69fb908a1a78483b62


This means that 3.12.1 should fix this problem, and that release
is supposedly "imminent": 3.12.0 was released on 2023-10-02 and
subsequent bugfix releases are scheduled every two months.


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191433): 
https://lists.openembedded.org/g/openembedded-core/message/191433
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-28 Thread Alexander Kanavin

On Tue, 28 Nov 2023 at 11:31, Zoltan Boszormenyi  wrote:
> The warning only occurs if os.fork() is called when
> there are already Python threads.
>
> I guess the general performance uplift in Python 3.12
> can trigger a pre-existing issue in bitbake more easily.
>
> It is entirely possible that the threads bitbake creates do not
> exist yet with Python 3.11 and earlier at the time of executing
> os.fork(). ¯\_(ツ)_/¯

I think you might be too quick to jump to conclusions here. I've fixed
the async issue I've been observing (backporting
https://github.com/python/cpython/commit/2e5d4e24ebcd671915427b6fd35a5b0eea412c4a
), and when running bitbake with 3.12 from buildtools-tarball,
everything works fine, and I'm not seeing any leftover processes. Poky
revision is current master:
https://git.yoctoproject.org/poky/commit/?h=master=3e50e45917831d9da2d86e69fb908a1a78483b62

Alex

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191379): 
https://lists.openembedded.org/g/openembedded-core/message/191379
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-28 Thread Zoltan Boszormenyi


2023. 11. 28. 11:00 keltezéssel, Richard Purdie írta:

On Tue, 2023-11-28 at 10:26 +0100, Böszörményi Zoltán wrote:

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that
it is indeed an issue even in C / POSIX: only the thread
executing fork() will become part of the new process,
but not the main process starting the thread or any
other threads from the original process.

The warning is justified, and apparently the issue
not occurring in bitbake with previous Python versions
is pure luck.

I'm not sure that is entirely true/fair.


Maybe not fair. Still, true. See this piece:

http://www.doublersolutions.com/docs/dce/osfdocs/htmls/develop/appdev/Appde193.htm

Other discussions, like this one confirm the same:
https://stackoverflow.com/questions/39890363/what-happens-when-a-thread-forks


  The way bitbake has executed
processes and handled threads was very carefully written deliberately
to avoid some of these kinds of issues so it wasn't "pure luck". What I
don't know is what changed in 3.12 to mean this now is potentially
breaking.


The warning only occurs if os.fork() is called when
there are already Python threads.

I guess the general performance uplift in Python 3.12
can trigger a pre-existing issue in bitbake more easily.

It is entirely possible that the threads bitbake creates do not
exist yet with Python 3.11 and earlier at the time of executing
os.fork(). ¯\_(ツ)_/¯


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191354): 
https://lists.openembedded.org/g/openembedded-core/message/191354
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-28 Thread Alexander Kanavin

On Tue, 28 Nov 2023 at 11:00, Richard Purdie
 wrote:
> I'm not sure that is entirely true/fair. The way bitbake has executed
> processes and handled threads was very carefully written deliberately
> to avoid some of these kinds of issues so it wasn't "pure luck". What I
> don't know is what changed in 3.12 to mean this now is potentially
> breaking.

With my python 3.12 recipe I'm getting a different issue when that
recipe is used to produce python in buildtools, and much sooner:
https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8593/steps/13/logs/stdio

I've reproduced it locally, and looking into it. It seems as though
hash server is not creating the unix socket file, but isn't failing to
start either.

Alex

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191352): 
https://lists.openembedded.org/g/openembedded-core/message/191352
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-28 Thread Richard Purdie

On Tue, 2023-11-28 at 10:26 +0100, Böszörményi Zoltán wrote:
> So it's not just Fedora, but Python 3.12 in general.
> 
> Googling for problem with threads + fork reveals that
> it is indeed an issue even in C / POSIX: only the thread
> executing fork() will become part of the new process,
> but not the main process starting the thread or any
> other threads from the original process.
> 
> The warning is justified, and apparently the issue
> not occurring in bitbake with previous Python versions
> is pure luck.

I'm not sure that is entirely true/fair. The way bitbake has executed
processes and handled threads was very carefully written deliberately
to avoid some of these kinds of issues so it wasn't "pure luck". What I
don't know is what changed in 3.12 to mean this now is potentially
breaking.

Cheers,

Richard





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191351): 
https://lists.openembedded.org/g/openembedded-core/message/191351
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-28 Thread Zoltan Boszormenyi


So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that
it is indeed an issue even in C / POSIX: only the thread
executing fork() will become part of the new process,
but not the main process starting the thread or any
other threads from the original process.

The warning is justified, and apparently the issue
not occurring in bitbake with previous Python versions
is pure luck.

2023. 11. 27. 14:14 keltezéssel, Martin Jansa írta:

I can confirm the same on gentoo with python-3.12.

On Mon, Nov 27, 2023 at 12:29 PM Zoltan Boszormenyi  wrote:

2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:
> Is there anything interesting in the bitbake-cookerdaemon.log file?

Actually, yes.

This is obviously the most suspicious:

/usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: 
This
process
(pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in the 
child.
   self.pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: 
DeprecationWarning: This
process
(pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in the 
child.
   pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: 
DeprecationWarning: This
process
(pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in the 
child.
   pid = os.fork()

There are a few other deprecation warnings, too,
which are not relevant to this particular problem
but may be interesting for the future, nevertheless:

/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260: 
DeprecationWarning:
ast.Str is
deprecated and will be removed in Python 3.14; use ast.Constant instead
   if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263: 
DeprecationWarning:
Attribute
s is deprecated and will be removed in Python 3.14; use value instead
   self.contains[varname].add(node.args[1].s)

ruby.bbclass has a lot of these at different line numbers:


/home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28:
SyntaxWarning: invalid escape sequence '\.'
   r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")

These lines occur at the end (probably normal):

1605202 10:39:58.854287 Parse cache valid
1605202 10:40:13.098694 Registering idle function .buildTargetsIdle at 0x7fa3e22e79c0>
1605202 10:40:13.098736 Removing idle function >
1605202 12:17:02.357378 Running command ['stateShutdown']
1605202 12:17:02.357622 Sending reply (None, None)
1605202 12:17:02.357819 Command Completed (socket: True)

And the remaining child processes:

$ ps auxw | grep bitb | grep -v grep
zozo 1605202  2.3  3.5 184 1151808 ? Sl   10:39 2:29 
/usr/bin/python3
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
/home/zozo/test-yocto-4.3/bitbake.lock
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
zozo 1605208  0.0  1.4 559372 466260 ?   S    10:39 0:00 
/usr/bin/python3
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
/home/zozo/test-yocto-4.3/bitbake.lock
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191349): 
https://lists.openembedded.org/g/openembedded-core/message/191349
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Martin Jansa

I can confirm the same on gentoo with python-3.12.

On Mon, Nov 27, 2023 at 12:29 PM Zoltan Boszormenyi 
wrote:

> 2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:
> > Is there anything interesting in the bitbake-cookerdaemon.log file?
>
> Actually, yes.
>
> This is obviously the most suspicious:
>
> /usr/lib64/python3.12/multiprocessing/popen_fork.py:66:
> DeprecationWarning: This process
> (pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in
> the child.
>self.pid = os.fork()
> /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210:
> DeprecationWarning: This process
> (pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in
> the child.
>pid = os.fork()
> /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210:
> DeprecationWarning: This process
> (pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in
> the child.
>pid = os.fork()
>
> There are a few other deprecation warnings, too,
> which are not relevant to this particular problem
> but may be interesting for the future, nevertheless:
>
> /home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260:
> DeprecationWarning: ast.Str is
> deprecated and will be removed in Python 3.14; use ast.Constant instead
>if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
> /home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263:
> DeprecationWarning: Attribute
> s is deprecated and will be removed in Python 3.14; use value instead
>self.contains[varname].add(node.args[1].s)
>
> ruby.bbclass has a lot of these at different line numbers:
>
> /home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28:
>
> SyntaxWarning: invalid escape sequence '\.'
>r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")
>
> These lines occur at the end (probably normal):
>
> 1605202 10:39:58.854287 Parse cache valid
> 1605202 10:40:13.098694 Registering idle function  BBCooker.buildTargets..buildTargetsIdle at 0x7fa3e22e79c0>
> 1605202 10:40:13.098736 Removing idle function  Command.runAsyncCommand of
> >
> 1605202 12:17:02.357378 Running command ['stateShutdown']
> 1605202 12:17:02.357622 Sending reply (None, None)
> 1605202 12:17:02.357819 Command Completed (socket: True)
>
> And the remaining child processes:
>
> $ ps auxw | grep bitb | grep -v grep
> zozo 1605202  2.3  3.5 184 1151808 ? Sl   10:39   2:29
> /usr/bin/python3
> /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
> /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
> /home/zozo/test-yocto-4.3/bitbake.lock
> /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
> zozo 1605208  0.0  1.4 559372 466260 ?   S10:39   0:00
> /usr/bin/python3
> /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
> /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
> /home/zozo/test-yocto-4.3/bitbake.lock
> /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191298): 
https://lists.openembedded.org/g/openembedded-core/message/191298
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Zoltan Boszormenyi


2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:

Is there anything interesting in the bitbake-cookerdaemon.log file?


Actually, yes.

This is obviously the most suspicious:

/usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process 
(pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in the child.

  self.pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process 
(pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in the child.

  pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process 
(pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in the child.

  pid = os.fork()

There are a few other deprecation warnings, too,
which are not relevant to this particular problem
but may be interesting for the future, nevertheless:

/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260: DeprecationWarning: ast.Str is 
deprecated and will be removed in Python 3.14; use ast.Constant instead

  if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263: DeprecationWarning: Attribute 
s is deprecated and will be removed in Python 3.14; use value instead

  self.contains[varname].add(node.args[1].s)

ruby.bbclass has a lot of these at different line numbers:

/home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28: 
SyntaxWarning: invalid escape sequence '\.'

  r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")

These lines occur at the end (probably normal):

1605202 10:39:58.854287 Parse cache valid
1605202 10:40:13.098694 Registering idle function BBCooker.buildTargets..buildTargetsIdle at 0x7fa3e22e79c0>
1605202 10:40:13.098736 Removing idle function >

1605202 12:17:02.357378 Running command ['stateShutdown']
1605202 12:17:02.357622 Sending reply (None, None)
1605202 12:17:02.357819 Command Completed (socket: True)

And the remaining child processes:

$ ps auxw | grep bitb | grep -v grep
zozo 1605202  2.3  3.5 184 1151808 ? Sl   10:39   2:29 /usr/bin/python3 
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock 
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
zozo 1605208  0.0  1.4 559372 466260 ?   S    10:39   0:00 /usr/bin/python3 
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock 
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191294): 
https://lists.openembedded.org/g/openembedded-core/message/191294
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Richard Purdie

On Mon, 2023-11-27 at 10:52 +0100, Zoltan Boszormenyi wrote:
> Hi,
> 
> I am not sure what's the actual problem here.
> 
> FWIW, I have recently upgraded my laptop to Fedora 39
> where I test recipe changes first.
> 
> Now, whenever a build ends (no matter if successfully or
> if there was a recipe build failure) two bitbake-server
> processes stay running.
> 
> As a result, the next bitbake run spits out messages about
> the previous instance holding bitbake.lock and retries.
> 
> Fedora 39 has Python 3.12 but maybe an outdated python
> module is the issue.
> 
> I am happy to try any ideas.

Is there anything interesting in the bitbake-cookerdaemon.log file?

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191291): 
https://lists.openembedded.org/g/openembedded-core/message/191291
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Alexander Kanavin

Fedora 39 isn't yet supported or tested, so it's almost certainly
you've hit a python 3.12 compatibility issue. If you can look into,
that would be very appreciated.

Alex

On Mon, 27 Nov 2023 at 10:52, Zoltan Boszormenyi  wrote:
>
> Hi,
>
> I am not sure what's the actual problem here.
>
> FWIW, I have recently upgraded my laptop to Fedora 39
> where I test recipe changes first.
>
> Now, whenever a build ends (no matter if successfully or
> if there was a recipe build failure) two bitbake-server
> processes stay running.
>
> As a result, the next bitbake run spits out messages about
> the previous instance holding bitbake.lock and retries.
>
> Fedora 39 has Python 3.12 but maybe an outdated python
> module is the issue.
>
> I am happy to try any ideas.
>
> Thanks in advance,
> Zoltán
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191290): 
https://lists.openembedded.org/g/openembedded-core/message/191290
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Zoltan Boszormenyi


2023. 11. 27. 10:52 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org 
írta:

Hi,

I am not sure what's the actual problem here.

FWIW, I have recently upgraded my laptop to Fedora 39
where I test recipe changes first.

Now, whenever a build ends (no matter if successfully or
if there was a recipe build failure) two bitbake-server
processes stay running.


Builds for mickledore and nanbield both show the same issue.



As a result, the next bitbake run spits out messages about
the previous instance holding bitbake.lock and retries.

Fedora 39 has Python 3.12 but maybe an outdated python
module is the issue.

I am happy to try any ideas.

Thanks in advance,
Zoltán







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191264): 
https://lists.openembedded.org/g/openembedded-core/message/191264
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] bitbake-server does not exit after build ends

2023-11-27 Thread Zoltan Boszormenyi


Hi,

I am not sure what's the actual problem here.

FWIW, I have recently upgraded my laptop to Fedora 39
where I test recipe changes first.

Now, whenever a build ends (no matter if successfully or
if there was a recipe build failure) two bitbake-server
processes stay running.

As a result, the next bitbake run spits out messages about
the previous instance holding bitbake.lock and retries.

Fedora 39 has Python 3.12 but maybe an outdated python
module is the issue.

I am happy to try any ideas.

Thanks in advance,
Zoltán


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191263): 
https://lists.openembedded.org/g/openembedded-core/message/191263
Mute This Topic: https://lists.openembedded.org/mt/102826186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

Re: [OE-core] bitbake-server does not exit after build ends

[OE-core] bitbake-server does not exit after build ends

22 matches

Site Navigation

Mail list logo

Footer information