Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-27 Thread Bruce Dubbs via blfs-dev

On 3/27/20 12:19 PM, Pierre Labastie via blfs-dev wrote:

On Fri, 2020-03-27 at 15:47 +, Ken Moffat via blfs-dev wrote:



If you wish to change what is in LFS I'll leave that
discussion/decision to those with a better understanding than I
have.



Agreed, we should have another thread for changing lfs on lfs-dev (or
not).


The only problem we've found is for Mozilla based packages.  LFS works 
as it is.  I do not think it needs to be changed.


  -- Bruce

--
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-27 Thread Pierre Labastie via blfs-dev
On Fri, 2020-03-27 at 15:47 +, Ken Moffat via blfs-dev wrote:
> On Fri, Mar 27, 2020 at 10:11:20PM +0800, Xi Ruoyao via blfs-dev
> wrote:
> > [...]
> 
> For me, I do not use my normal script for entering chroot when I'm
> building a desktop in chroot.  Whenever I do that I've normally done
> a boot test, and then removed or renamed /tools, so invoking
> /tools/bin/env etc doesn't work.

I use the commands at the end of chapter 6 (Cleaning up). They do not
refer to /tools. Anyway, I agree with what you say below.

> 
> Anyway, I'm not concerned with whether or not the commands for
> entering chroot in the LFS book should be changed - I expect to be
> able to use the process I mentioned earlier (e.g. when using a rescue
> stick) to get into a completed system, and I use the same approach
> when building packages in chroot.
> 
> And so far it is only these mozilla derivatives which need shm
> during their configure stages.
> 
> If you wish to change what is in LFS I'll leave that
> discussion/decision to those with a better understanding than I
> have.
> 

Agreed, we should have another thread for changing lfs on lfs-dev (or
not).

Pierre

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-27 Thread Ken Moffat via blfs-dev
On Fri, Mar 27, 2020 at 10:11:20PM +0800, Xi Ruoyao via blfs-dev wrote:
> On 2020-03-27 11:46 +, Ken Moffat via blfs-dev wrote:
> > On Fri, Mar 27, 2020 at 07:03:29AM +0100, Pierre Labastie via blfs-dev 
> > wrote:
> > > Le 27/03/2020 à 00:22, Ken Moffat via blfs-dev a écrit :
> > > > On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev
> > > > wrote:
> > > > 
> > > > (asking about only one item)
> > > > > > If you do not do this, configuring will fail with a python traceback
> > > > > > report referencing a 
> > > > > > /usr/lib/pythonN.N/multiprocessing/synchronize.py
> > > > > > file and ending 'OSError: [Errno 38] Function not implemented'.
> > > > > > (this explanation possibly in italics, i.e. emphasis, except for the
> > > > > > filename markup)
> > > > > 
> > > > > As the starter of this thread, I do not see exactly this error, but
> > > > > rather:
> > > > > 
> > > > > File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in
> > > > > 
> > > > > " function, see issue 3770.")
> > > > > ImportError: This platform lacks a functioning sem_open 
> > > > > implementation,
> > > > > therefore, the required synchronization primitives needed will not
> > > > > function,
> > > > > see issue 3770.
> > > > > 
> > > > 
> > > > Which package is this which does not mention OSError, please ?
> > > > 
> > > 
> > > Hmmm, I realize that "as the starter of the thread" may mean I started the
> > > thread... Of course I did not. It is the message you mentioned earlier 
> > > that
> > > started that (from Nagasayanam, V.S. on March 21st). The message is about
> > > mozjs-60.8.0. And you can see that there is no mention of OSError.
> > 
> > Thanks.  That was why it took me so long to originally find my notes
> > (for which the key item to grep was OSError).  I assumed it might
> > have been trimmed off.
> > 
> > > When trying today with host debian, I get:
> > > --
> > >  0:23.64 Creating config.status
> > >  0:23.76 Traceback (most recent call last):
> > >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 132, 
> > > in
> > > 
> > >  0:23.76 sys.exit(main(sys.argv))
> > >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 43, 
> > > in
> > > main
> > >  0:23.76 return config_status(config)
> > >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 127, 
> > > in
> > > config_status
> > >  0:23.76 return config_status(args=[], **encode(sanitized_config,
> > > encoding))
> > >  0:23.76   File
> > > "/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/config_status.py",
> > > line 132, in config_status
> > >  0:23.76 reader = BuildReader(env)
> > >  0:23.76   File
> > > "/sources/firefox/firefox-
> > > 68.6.0/python/mozbuild/mozbuild/frontend/reader.py",
> > > line 854, in __init__
> > >  0:23.77 self._gyp_worker_pool =
> > > ProcessPoolExecutor(max_workers=max_workers)
> > >  0:23.77   File
> > > "/sources/firefox/firefox-
> > > 68.6.0/third_party/python/futures/concurrent/futures/process.py",
> > > line 285, in __init__
> > >  0:23.77 EXTRA_QUEUED_CALLS)
> > >  0:23.77   File "/usr/lib/python2.7/multiprocessing/__init__.py", line 
> > > 218,
> > > in
> > > Queue
> > >  0:23.77 return Queue(maxsize)
> > >  0:23.77   File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, 
> > > in
> > > __init__
> > >  0:23.77 self._rlock = Lock()
> > >  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line
> > > 147,
> > > in __init__
> > >  0:23.77 SemLock.__init__(self, SEMAPHORE, 1, 1)
> > >  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line
> > > 75,
> > > in __init__
> > >  0:23.77 sl = self._semlock = _multiprocessing.SemLock(kind, value,
> > > maxvalue)
> > >  0:23.77 OSError: [Errno 13] Permission denied
> > >  0:23.84 *** Fix above errors and then restart with\
> > >  0:23.84"./mach build"
> > >  0:23.84 make: *** [client.mk:115: configure] Error 1
> > > 
> > > So again different.
> > 
> > OK, Every example mentions multiprocessing/synchronize.py near the
> > end, but the exact details of the error message vary.
> > 
> > I'll drop the
> >   and ending 'OSError: [Errno 38] Function not implemented'
> > text.
> > 
> > I guess debian is passing back -EPERM from an attempt to access shm,
> > whereas when LFS is the host that gets replaced by Error 38.
> > 
> > > Note that:
> > > mount -t tmpfs devshm /dev/shm
> > > cures the error
> > > 
> > > Pierre
> > 
> > I can settle for that, but was this build on sysvinit ?  From
> > xry111's reply, and your earlier points, I got the impression this
> > is only a problem on BLFS sysvinit.
> > 
> > If so, the Note should only be added in the sysvinit book.
> 
> No.  It's only *not* a problem if the host is LFS-sysvinit or debian.  They 
> uses
> /dev/shm -> /run/shm and we explicitly handle this in LFS section 6.2.
> 

For me, I do not use my normal script for entering chroot when

Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-27 Thread Xi Ruoyao via blfs-dev
On 2020-03-27 11:46 +, Ken Moffat via blfs-dev wrote:
> On Fri, Mar 27, 2020 at 07:03:29AM +0100, Pierre Labastie via blfs-dev wrote:
> > Le 27/03/2020 à 00:22, Ken Moffat via blfs-dev a écrit :
> > > On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev
> > > wrote:
> > > 
> > > (asking about only one item)
> > > > > If you do not do this, configuring will fail with a python traceback
> > > > > report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
> > > > > file and ending 'OSError: [Errno 38] Function not implemented'.
> > > > > (this explanation possibly in italics, i.e. emphasis, except for the
> > > > > filename markup)
> > > > 
> > > > As the starter of this thread, I do not see exactly this error, but
> > > > rather:
> > > > 
> > > > File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in
> > > > 
> > > > " function, see issue 3770.")
> > > > ImportError: This platform lacks a functioning sem_open implementation,
> > > > therefore, the required synchronization primitives needed will not
> > > > function,
> > > > see issue 3770.
> > > > 
> > > 
> > > Which package is this which does not mention OSError, please ?
> > > 
> > 
> > Hmmm, I realize that "as the starter of the thread" may mean I started the
> > thread... Of course I did not. It is the message you mentioned earlier that
> > started that (from Nagasayanam, V.S. on March 21st). The message is about
> > mozjs-60.8.0. And you can see that there is no mention of OSError.
> 
> Thanks.  That was why it took me so long to originally find my notes
> (for which the key item to grep was OSError).  I assumed it might
> have been trimmed off.
> 
> > When trying today with host debian, I get:
> > --
> >  0:23.64 Creating config.status
> >  0:23.76 Traceback (most recent call last):
> >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 132, in
> > 
> >  0:23.76 sys.exit(main(sys.argv))
> >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 43, in
> > main
> >  0:23.76 return config_status(config)
> >  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 127, in
> > config_status
> >  0:23.76 return config_status(args=[], **encode(sanitized_config,
> > encoding))
> >  0:23.76   File
> > "/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/config_status.py",
> > line 132, in config_status
> >  0:23.76 reader = BuildReader(env)
> >  0:23.76   File
> > "/sources/firefox/firefox-
> > 68.6.0/python/mozbuild/mozbuild/frontend/reader.py",
> > line 854, in __init__
> >  0:23.77 self._gyp_worker_pool =
> > ProcessPoolExecutor(max_workers=max_workers)
> >  0:23.77   File
> > "/sources/firefox/firefox-
> > 68.6.0/third_party/python/futures/concurrent/futures/process.py",
> > line 285, in __init__
> >  0:23.77 EXTRA_QUEUED_CALLS)
> >  0:23.77   File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218,
> > in
> > Queue
> >  0:23.77 return Queue(maxsize)
> >  0:23.77   File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in
> > __init__
> >  0:23.77 self._rlock = Lock()
> >  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line
> > 147,
> > in __init__
> >  0:23.77 SemLock.__init__(self, SEMAPHORE, 1, 1)
> >  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line
> > 75,
> > in __init__
> >  0:23.77 sl = self._semlock = _multiprocessing.SemLock(kind, value,
> > maxvalue)
> >  0:23.77 OSError: [Errno 13] Permission denied
> >  0:23.84 *** Fix above errors and then restart with\
> >  0:23.84"./mach build"
> >  0:23.84 make: *** [client.mk:115: configure] Error 1
> > 
> > So again different.
> 
> OK, Every example mentions multiprocessing/synchronize.py near the
> end, but the exact details of the error message vary.
> 
> I'll drop the
>   and ending 'OSError: [Errno 38] Function not implemented'
> text.
> 
> I guess debian is passing back -EPERM from an attempt to access shm,
> whereas when LFS is the host that gets replaced by Error 38.
> 
> > Note that:
> > mount -t tmpfs devshm /dev/shm
> > cures the error
> > 
> > Pierre
> 
> I can settle for that, but was this build on sysvinit ?  From
> xry111's reply, and your earlier points, I got the impression this
> is only a problem on BLFS sysvinit.
> 
> If so, the Note should only be added in the sysvinit book.

No.  It's only *not* a problem if the host is LFS-sysvinit or debian.  They uses
/dev/shm -> /run/shm and we explicitly handle this in LFS section 6.2.

For other hosts, /dev/shm is a mount point of a seperated tmpfs, with perm 0777.
In mount -v --bind /dev $LFS/dev, --bind is not recursive so $LFS/dev/shm
becames a empty directory, owned by root:root with perm 0755.  So normal users
won't be able to use POSIX shared memory and semaphore.

I think we should change the command in 6.2:

if [ -h $LFS/dev/shm ]; then
  mkdir -pv $LFS/$(readlink $LFS/dev/shm)
fi

to

mount -v -t tmpfs dev

Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-27 Thread Ken Moffat via blfs-dev
On Fri, Mar 27, 2020 at 07:03:29AM +0100, Pierre Labastie via blfs-dev wrote:
> Le 27/03/2020 à 00:22, Ken Moffat via blfs-dev a écrit :
> > On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev 
> > wrote:
> > 
> > (asking about only one item)
> >>
> >>>
> >>> If you do not do this, configuring will fail with a python traceback
> >>> report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
> >>> file and ending 'OSError: [Errno 38] Function not implemented'.
> >>> (this explanation possibly in italics, i.e. emphasis, except for the
> >>> filename markup)
> >>
> >> As the starter of this thread, I do not see exactly this error, but rather:
> >>
> >> File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in 
> >> 
> >> " function, see issue 3770.")
> >> ImportError: This platform lacks a functioning sem_open implementation,
> >> therefore, the required synchronization primitives needed will not 
> >> function,
> >> see issue 3770.
> >>
> > 
> > Which package is this which does not mention OSError, please ?
> > 
> 
> Hmmm, I realize that "as the starter of the thread" may mean I started the
> thread... Of course I did not. It is the message you mentioned earlier that
> started that (from Nagasayanam, V.S. on March 21st). The message is about
> mozjs-60.8.0. And you can see that there is no mention of OSError.

Thanks.  That was why it took me so long to originally find my notes
(for which the key item to grep was OSError).  I assumed it might
have been trimmed off.

> When trying today with host debian, I get:
> --
>  0:23.64 Creating config.status
>  0:23.76 Traceback (most recent call last):
>  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 132, in
> 
>  0:23.76 sys.exit(main(sys.argv))
>  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 43, in 
> main
>  0:23.76 return config_status(config)
>  0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 127, in
> config_status
>  0:23.76 return config_status(args=[], **encode(sanitized_config, 
> encoding))
>  0:23.76   File
> "/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/config_status.py",
> line 132, in config_status
>  0:23.76 reader = BuildReader(env)
>  0:23.76   File
> "/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/frontend/reader.py",
> line 854, in __init__
>  0:23.77 self._gyp_worker_pool = 
> ProcessPoolExecutor(max_workers=max_workers)
>  0:23.77   File
> "/sources/firefox/firefox-68.6.0/third_party/python/futures/concurrent/futures/process.py",
> line 285, in __init__
>  0:23.77 EXTRA_QUEUED_CALLS)
>  0:23.77   File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218, in
> Queue
>  0:23.77 return Queue(maxsize)
>  0:23.77   File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in
> __init__
>  0:23.77 self._rlock = Lock()
>  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147,
> in __init__
>  0:23.77 SemLock.__init__(self, SEMAPHORE, 1, 1)
>  0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75,
> in __init__
>  0:23.77 sl = self._semlock = _multiprocessing.SemLock(kind, value, 
> maxvalue)
>  0:23.77 OSError: [Errno 13] Permission denied
>  0:23.84 *** Fix above errors and then restart with\
>  0:23.84"./mach build"
>  0:23.84 make: *** [client.mk:115: configure] Error 1
> 
> So again different.

OK, Every example mentions multiprocessing/synchronize.py near the
end, but the exact details of the error message vary.

I'll drop the
  and ending 'OSError: [Errno 38] Function not implemented'
text.

I guess debian is passing back -EPERM from an attempt to access shm,
whereas when LFS is the host that gets replaced by Error 38.

> Note that:
> mount -t tmpfs devshm /dev/shm
> cures the error
> 
> Pierre

I can settle for that, but was this build on sysvinit ?  From
xry111's reply, and your earlier points, I got the impression this
is only a problem on BLFS sysvinit.

If so, the Note should only be added in the sysvinit book.

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 27/03/2020 à 00:22, Ken Moffat via blfs-dev a écrit :
> On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev wrote:
> 
> (asking about only one item)
>>
>>>
>>> If you do not do this, configuring will fail with a python traceback
>>> report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
>>> file and ending 'OSError: [Errno 38] Function not implemented'.
>>> (this explanation possibly in italics, i.e. emphasis, except for the
>>> filename markup)
>>
>> As the starter of this thread, I do not see exactly this error, but rather:
>>
>> File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in 
>> 
>> " function, see issue 3770.")
>> ImportError: This platform lacks a functioning sem_open implementation,
>> therefore, the required synchronization primitives needed will not function,
>> see issue 3770.
>>
> 
> Which package is this which does not mention OSError, please ?
> 

Hmmm, I realize that "as the starter of the thread" may mean I started the
thread... Of course I did not. It is the message you mentioned earlier that
started that (from Nagasayanam, V.S. on March 21st). The message is about
mozjs-60.8.0. And you can see that there is no mention of OSError.
When trying today with host debian, I get:
--
 0:23.64 Creating config.status
 0:23.76 Traceback (most recent call last):
 0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 132, in

 0:23.76 sys.exit(main(sys.argv))
 0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 43, in main
 0:23.76 return config_status(config)
 0:23.76   File "/sources/firefox/firefox-68.6.0/configure.py", line 127, in
config_status
 0:23.76 return config_status(args=[], **encode(sanitized_config, encoding))
 0:23.76   File
"/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/config_status.py",
line 132, in config_status
 0:23.76 reader = BuildReader(env)
 0:23.76   File
"/sources/firefox/firefox-68.6.0/python/mozbuild/mozbuild/frontend/reader.py",
line 854, in __init__
 0:23.77 self._gyp_worker_pool = 
ProcessPoolExecutor(max_workers=max_workers)
 0:23.77   File
"/sources/firefox/firefox-68.6.0/third_party/python/futures/concurrent/futures/process.py",
line 285, in __init__
 0:23.77 EXTRA_QUEUED_CALLS)
 0:23.77   File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218, in
Queue
 0:23.77 return Queue(maxsize)
 0:23.77   File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in
__init__
 0:23.77 self._rlock = Lock()
 0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147,
in __init__
 0:23.77 SemLock.__init__(self, SEMAPHORE, 1, 1)
 0:23.77   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75,
in __init__
 0:23.77 sl = self._semlock = _multiprocessing.SemLock(kind, value, 
maxvalue)
 0:23.77 OSError: [Errno 13] Permission denied
 0:23.84 *** Fix above errors and then restart with\
 0:23.84"./mach build"
 0:23.84 make: *** [client.mk:115: configure] Error 1

So again different. Note that:
mount -t tmpfs devshm /dev/shm
cures the error

Pierre
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Bruce Dubbs via blfs-dev

On 3/26/20 8:38 PM, Xi Ruoyao via blfs-dev wrote:

On 2020-03-26 23:22 +, Ken Moffat via blfs-dev wrote:

On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev wrote:

(asking about only one item)

If you do not do this, configuring will fail with a python traceback
report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
file and ending 'OSError: [Errno 38] Function not implemented'.
(this explanation possibly in italics, i.e. emphasis, except for the
filename markup)


As the starter of this thread, I do not see exactly this error, but rather:

File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in

 " function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation,
therefore, the required synchronization primitives needed will not function,
see issue 3770.



from tmpfs (5):


A tmpfs filesystem mounted at /dev/shm is used for  the  implementation
of   POSIX   shared   memory  (shm_overview(7))  and  POSIX  semaphores
(sem_overview(7)).


In kernel doc (Documentation/filesystems/tmpfs.txt):


2) glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for
POSIX shared memory (shm_open, shm_unlink). Adding the following
line to /etc/fstab should take care of this:

 tmpfs   /dev/shmtmpfs   defaults0 0

Remember to create the directory that you intend to mount tmpfs on
if necessary.

This mount is _not_ needed for SYSV shared memory. The internal
mount is used for that. (In the 2.3 kernel versions it was
necessary to mount the predecessor of tmpfs (shm fs) to use SYSV
shared memory)


So I think we should mount a tmpfs to /dev/shm, anyway.

In lfs-systemd it's automatically done by systemd.  In lfs-sysv book the
mountvirtfs bootscript makes /run/shm and link it to /dev/shm.  I think it's a
"hack" which is deprecated, and shall be replaced with a line mounting a tmpfs
to /dev/shm, in /etc/fstab.


If /dev/shm is a symlink to /run/shm and /run is a tmpfs and shm is a 
subdirectory of /run, what is the difference?  There is one level of 
indirection with the symlink, but any operations end up on a tmpfs now.


  -- Bruce

--
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Xi Ruoyao via blfs-dev
On 2020-03-26 23:22 +, Ken Moffat via blfs-dev wrote:
> On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev wrote:
> 
> (asking about only one item)
> > > If you do not do this, configuring will fail with a python traceback
> > > report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
> > > file and ending 'OSError: [Errno 38] Function not implemented'.
> > > (this explanation possibly in italics, i.e. emphasis, except for the
> > > filename markup)
> > 
> > As the starter of this thread, I do not see exactly this error, but rather:
> > 
> > File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in
> > 
> > " function, see issue 3770.")
> > ImportError: This platform lacks a functioning sem_open implementation,
> > therefore, the required synchronization primitives needed will not function,
> > see issue 3770.
> > 

from tmpfs (5):

>A tmpfs filesystem mounted at /dev/shm is used for  the  implementation
>of   POSIX   shared   memory  (shm_overview(7))  and  POSIX  semaphores
>(sem_overview(7)).

In kernel doc (Documentation/filesystems/tmpfs.txt):

> 2) glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for
>POSIX shared memory (shm_open, shm_unlink). Adding the following
>line to /etc/fstab should take care of this:
> 
> tmpfs   /dev/shmtmpfs   defaults0 0
> 
>Remember to create the directory that you intend to mount tmpfs on
>if necessary.
> 
>This mount is _not_ needed for SYSV shared memory. The internal
>mount is used for that. (In the 2.3 kernel versions it was
>necessary to mount the predecessor of tmpfs (shm fs) to use SYSV
>shared memory)

So I think we should mount a tmpfs to /dev/shm, anyway.

In lfs-systemd it's automatically done by systemd.  In lfs-sysv book the
mountvirtfs bootscript makes /run/shm and link it to /dev/shm.  I think it's a
"hack" which is deprecated, and shall be replaced with a line mounting a tmpfs
to /dev/shm, in /etc/fstab.

> Which package is this which does not mention OSError, please ?
> 
> ĸen
> -- 
> When alle is ſayed and all is done, ye must chooſe your faces wisely,
> for soon enouff ye will be playing with fyre."
>   The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Ken Moffat via blfs-dev
On Thu, Mar 26, 2020 at 10:57:03PM +0100, Pierre Labastie via blfs-dev wrote:

(asking about only one item)
> 
> > 
> > If you do not do this, configuring will fail with a python traceback
> > report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
> > file and ending 'OSError: [Errno 38] Function not implemented'.
> > (this explanation possibly in italics, i.e. emphasis, except for the
> > filename markup)
> 
> As the starter of this thread, I do not see exactly this error, but rather:
> 
> File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in 
> " function, see issue 3770.")
> ImportError: This platform lacks a functioning sem_open implementation,
> therefore, the required synchronization primitives needed will not function,
> see issue 3770.
> 

Which package is this which does not mention OSError, please ?

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 26/03/2020 à 18:15, Ken Moffat via blfs-dev a écrit :
> On Thu, Mar 26, 2020 at 04:51:59PM +, Ken Moffat via blfs-dev wrote:
>> On Thu, Mar 26, 2020 at 01:41:40PM +, Ken Moffat via blfs-dev wrote:
>>
>> [snip how, and what, I tested]
>>
> 
> And now the proposal:
> 
> Replace the Note about $SHELL and /dev/shm in firefox with a standard
> copy member. The mount part comes first because OSError 38 comes
> before any eventual failure to determine the shell.
> 
> 
> Note:
> 
> If you are compiling this package in chroot you must do two things:
> 
> First, as the root user, ensure that /run is mounted:
> 
>  mount -t tmpfs /run

But this _should_ already be done if the commands on the "Preparing the
Virtual Kernel File Systems" page have been run. Mounting before or after
entering chroot does not change anything (it is possible I am wrong with this,
but I need to be able to reproduce this on my systems to be convinced)

And this is not enough if the host does not have /dev/shm->/run/shm, as is the
case on debian and lfs systemd. On those systems, the note about /dev/shm _is_
accurate (well, not quite, it should be "ensure that a tempfs is mounted on
/dev/shm")

> 
> If you do not do this, configuring will fail with a python traceback
> report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
> file and ending 'OSError: [Errno 38] Function not implemented'.
> (this explanation possibly in italics, i.e. emphasis, except for the
> filename markup)

As the starter of this thread, I do not see exactly this error, but rather:

File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 59, in 
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation,
therefore, the required synchronization primitives needed will not function,
see issue 3770.

> 
> Second, as your normal user either set the $SHELL environment
> variable or prepend SHELL=/bin/sh.

Actually, the SHELL variable _is_ set (at least for my systems) for root in
chroot, but not exported. So "export SHELL" is enough. Then when doing su -
, the SHELL is set and exported for the user (note that I have the bash
shell startup files as in the blfs book).

> 
> 
> I suppose a name like xincludes/mozilla-chroot.xml
> 
> Use in firefox, mozjs60, mozjs68, seamonkey, thunderbird.

Actually, lfs sysv is somewhat weird among distros: it seems to be the only
distro with /dev/shm->/run/shm. All systemd distros have the other way around,
and gentoo and artix do not even have /run/shm.

I know it used to be common that distros had /dev/shm->/run/shm, but nowadays,
it seems to be not used anymore.

Pierre
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Bruce Dubbs via blfs-dev

On 3/26/20 3:29 PM, Pierre Labastie via blfs-dev wrote:

Le 26/03/2020 à 21:19, Pierre Labastie via blfs-dev a écrit :

Le 26/03/2020 à 18:32, Bruce Dubbs via blfs-dev a écrit :

On 3/26/20 10:22 AM, Xi Ruoyao via blfs-dev wrote:


Now the book is handling the case "/dev/shm is a link to /run/shm" correctly
and
explicitly.  Unfortunately it seems for the case "/dev/shm is *not* a link to
/run/shm" the book is defective.  I'm surprised that it didn't cause any
problem
building LFS...


I have not tested completely, but what if we did this in LFS instead of a bind
mount of $LFS/dev:

  mount -vt devtmpfs devtmpfs $LFS/dev

On my LFS system I then have:
  /mnt/lfs/dev/shm -> /run/shm

but if the host does not have  /mnt/lfs/dev/shm, then a simple
  mkdir -pv $LFS/dev/shm

should suffice.

   -- Bruce



Hmm, I have a problem with /dev/shm->/run/shm...
It is not true on lfs systemd!

All I've said is with lfs systemd






or debian (which is systemd too).


If you just do 'mount -vt devtmpfs devtmpfs $LFS/dev', do you have any 
$LFS/dev/shm ?


  -- Bruce
--
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 26/03/2020 à 21:19, Pierre Labastie via blfs-dev a écrit :
> Le 26/03/2020 à 18:32, Bruce Dubbs via blfs-dev a écrit :
>> On 3/26/20 10:22 AM, Xi Ruoyao via blfs-dev wrote:
>>
>>> Now the book is handling the case "/dev/shm is a link to /run/shm" correctly
>>> and
>>> explicitly.  Unfortunately it seems for the case "/dev/shm is *not* a link 
>>> to
>>> /run/shm" the book is defective.  I'm surprised that it didn't cause any
>>> problem
>>> building LFS...
>>
>> I have not tested completely, but what if we did this in LFS instead of a 
>> bind
>> mount of $LFS/dev:
>>
>>  mount -vt devtmpfs devtmpfs $LFS/dev
>>
>> On my LFS system I then have:
>>  /mnt/lfs/dev/shm -> /run/shm
>>
>> but if the host does not have  /mnt/lfs/dev/shm, then a simple
>>  mkdir -pv $LFS/dev/shm
>>
>> should suffice.
>>
>>   -- Bruce
>>
> 
> Hmm, I have a problem with /dev/shm->/run/shm...
> It is not true on lfs systemd!
> 
> All I've said is with lfs systemd

> 

or debian (which is systemd too).

Pierre
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 26/03/2020 à 18:32, Bruce Dubbs via blfs-dev a écrit :
> On 3/26/20 10:22 AM, Xi Ruoyao via blfs-dev wrote:
> 
>> Now the book is handling the case "/dev/shm is a link to /run/shm" correctly
>> and
>> explicitly.  Unfortunately it seems for the case "/dev/shm is *not* a link to
>> /run/shm" the book is defective.  I'm surprised that it didn't cause any
>> problem
>> building LFS...
> 
> I have not tested completely, but what if we did this in LFS instead of a bind
> mount of $LFS/dev:
> 
>  mount -vt devtmpfs devtmpfs $LFS/dev
> 
> On my LFS system I then have:
>  /mnt/lfs/dev/shm -> /run/shm
> 
> but if the host does not have  /mnt/lfs/dev/shm, then a simple
>  mkdir -pv $LFS/dev/shm
> 
> should suffice.
> 
>   -- Bruce
> 

Hmm, I have a problem with /dev/shm->/run/shm...
It is not true on lfs systemd!

All I've said is with lfs systemd
Pierre
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Bruce Dubbs via blfs-dev

On 3/26/20 10:22 AM, Xi Ruoyao via blfs-dev wrote:


Now the book is handling the case "/dev/shm is a link to /run/shm" correctly and
explicitly.  Unfortunately it seems for the case "/dev/shm is *not* a link to
/run/shm" the book is defective.  I'm surprised that it didn't cause any problem
building LFS...


I have not tested completely, but what if we did this in LFS instead of 
a bind mount of $LFS/dev:


 mount -vt devtmpfs devtmpfs $LFS/dev

On my LFS system I then have:
 /mnt/lfs/dev/shm -> /run/shm

but if the host does not have  /mnt/lfs/dev/shm, then a simple
 mkdir -pv $LFS/dev/shm

should suffice.

  -- Bruce

--
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Ken Moffat via blfs-dev
On Thu, Mar 26, 2020 at 04:51:59PM +, Ken Moffat via blfs-dev wrote:
> On Thu, Mar 26, 2020 at 01:41:40PM +, Ken Moffat via blfs-dev wrote:
> 
> [snip how, and what, I tested]
> 

And now the proposal:

Replace the Note about $SHELL and /dev/shm in firefox with a standard
copy member. The mount part comes first because OSError 38 comes
before any eventual failure to determine the shell.


Note:

If you are compiling this package in chroot you must do two things:

First, as the root user, ensure that /run is mounted:

 mount -t tmpfs /run

If you do not do this, configuring will fail with a python traceback
report referencing a /usr/lib/pythonN.N/multiprocessing/synchronize.py
file and ending 'OSError: [Errno 38] Function not implemented'.
(this explanation possibly in italics, i.e. emphasis, except for the
filename markup)

Second, as your normal user either set the $SHELL environment
variable or prepend SHELL=/bin/sh.


I suppose a name like xincludes/mozilla-chroot.xml

Use in firefox, mozjs60, mozjs68, seamonkey, thunderbird.

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Ken Moffat via blfs-dev
On Thu, Mar 26, 2020 at 01:41:40PM +, Ken Moffat via blfs-dev wrote:

[snip who replied to whom]

Here is what I did for testing.

> I'll reply in a while with what I have now tested (after I've
> assembled my notes).  I've gone with Bruce's suggestion to (only)
> mount a tmpfs on /run when building in chroot - I think xry111 has
> identified the issue (bind mounts are not recursive).
> 
More likely, it is the fact that in BLFS it is a symlink to
/run/shm.  Anyway, here is what I did and the errors.  Sorry, it is
long.  I'll post my proposed changes as a follow-up.

I think that in the past I noted that /dev/shm existed in chroot,
and since I don't often build these packages in chroot I assumed
that part was OK.

The testing:
===

For each package: mount a system at /mnt/lfs (host is 9.1, chroot is
LFS from 20200127 with BLFS from then and later)

bind /dev /proc /sys /sources /home
chroot
su - lfs

Attempt to build as user lfs.

ALL of these mozilla-derived packages failed.
On failure:
 note the message
 exit back to root (cannot su, not a tty)
   mount -t tmpfs /run
su - lfs

Successful retry with DESTDIR install

review contents of /run

exit back to root

umount /run

review contents of /run

exit chroot

umount /mnt/lfs and everything mounted on it :
 mount | grep lfs | awk '{ print $3 }' | tac | xargs umount

repeat for next package.

mozjs60:
---

(I initially tried strace during the configure, but I could not find
anything useful in the enormous output.)

Failed (as they all did) while creating config.status -

Traceback (most recent call last):
  File "../js/src/../../configure.py", line 127, in 
sys.exit(main(sys.argv))
  File "../js/src/../../configure.py", line 34, in main
return config_status(config)
  File "../js/src/../../configure.py", line 122, in config_status
return config_status(args=[], **encode(sanitized_config, encoding))
  File "/home/lfs/mozjs-60.8.0/python/mozbuild/mozbuild/config_status.py", line 
133, in config_status
reader = BuildReader(env)
  File "/home/lfs/mozjs-60.8.0/python/mozbuild/mozbuild/frontend/reader.py", 
line 868, in __init__
self._gyp_worker_pool = ProcessPoolExecutor(max_workers=max_workers)
  File 
"/home/lfs/mozjs-60.8.0/third_party/python/futures/concurrent/futures/process.py",
 line 285, in __init__
EXTRA_QUEUED_CALLS)
  File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218, in Queue
return Queue(maxsize)
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in __init__
self._rlock = Lock()
  File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147, in 
__init__
SemLock.__init__(self, SEMAPHORE, 1, 1)
  File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__
sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 38] Function not implemented

After successful build, /run was empty, but after I had umounted it,
while still in chroot I looked again : /run/mount and /run/shm
existed.

seamonkey:
-

The traceback ends:
  File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__
sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 38] Function not implemented
*** Fix above errors and then restart with\
   "make -f client.mk build"
make[1]: *** [/home/lfs/seamonkey-2.53.1/client.mk:362: configure] Error 1
make[1]: Leaving directory '/home/lfs/seamonkey-2.53.1'
make: *** [client.mk:374: 
/home/lfs/seamonkey-2.53.1/obj-x86_64-pc-linux-gnu/Makefile] Error 2

Looking at /run, empty both before umounting and afterwards


thunderbird:
---

 0:18.73   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, 
in __init__
 0:18.73 sl = self._semlock = _multiprocessing.SemLock(kind, value, 
maxvalue)
 0:18.73 OSError: [Errno 38] Function not implemented
 0:18.79 *** Fix above errors and then restart with\
 0:18.79"./mach build"
 0:18.79 make: *** [client.mk:115: configure] Error 1

NB, client.mk is invoked from ./mach.

Again. /run seemed to be empty even after unmounting.

firefox:
---

 0:12.67   File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, 
in __init__
 0:12.67 sl = self._semlock = _multiprocessing.SemLock(kind, value, 
maxvalue)
 0:12.67 OSError: [Errno 38] Function not implemented
  with message about fixing, restart with ./mach build
 [snipped, messages like thunderbird ]

After umount, /run contained /run/mount (only).

JS68:


  File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 38] Function not implemented

Again, after umount /run contained /run/mount (only).

Tested, but does NOT require this - node-js:
===

When I first encountered this problem (years ago, in firefox) for
some reason I thought it would be a good idea to rebuild node-js
although th

Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Ken Moffat via blfs-dev
On Thu, Mar 26, 2020 at 04:06:46PM +0100, Pierre Labastie via blfs-dev wrote:
> Le 26/03/2020 à 14:41, Ken Moffat via blfs-dev a écrit :
> > On Thu, Mar 26, 2020 at 11:40:59AM +0100, Pierre Labastie via
> > blfs-dev wrote:
> >> Le 23/03/2020 à 03:49, Bruce Dubbs via blfs-dev a écrit :
> >>> On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
>  On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> > On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via
> > blfs-dev wrote:
> >> On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> >>
[...]
> > I'll reply in a while with what I have now tested (after I've
> > assembled my notes).  I've gone with Bruce's suggestion to (only)
> > mount a tmpfs on /run when building in chroot - I think xry111 has
> > identified the issue (bind mounts are not recursive).
> > 
> > Meanwhile, if you do not see that OSError when building any of firefox,
> > mozjs{60,68}, seamonkey, thunderbird *in chroot* then you are doing
> > something different.  This issue was raised in
> > http://lists.linuxfromscratch.org/pipermail/blfs-support/2020-March/081897.html

Replying to this before I post my test results, and before I've
ready xry111's latest post.

Since I wrote that, I actually read the Note in firefox about
ensuring that /dev/shm is mounted.  ISTR that you cannot mount
/dev/shm in LFS because it is a broken symlink to /run/shm.

Therefore, I think the /dev/shm part of the note is old and
no-longer correct.

> 
> The error reported by the OP ("This platform lacks a functioning sem_open
> implementation...") is the one I see when /dev/shm is not world writable, and
> the build is done as a regular user.

Note that I am NOT touching /dev/shm directly.

> When building as root, the error is different (something about an unknown 
> shell).
> 

In my tests I initially (mozjs60) made sure SHELL was exported.  As
you will see when I post the details, I ended up exiting from chroot
and unmounting /mnt/lfs after successfully completing the tests for
each package.  Later it occurred to me that I had not exported SHELL
while testign the subsequent packages (second package was seahorse).
So I started another test, and was surprised to see that SHELL was
still in my lfs user's enviromnet.  I had mounted /run (known to be
necessary), so I unset SHELL.  Configure again failed while creating
config.log, this time because it could not determine the shell - and
that was as a regular user.

> Those are the only errors I see. Note that chroot is entered with the
> following commands (Makefile syntax):
>   sudo mount -v --bind /dev $(MOUNT_PT)/dev; \
> sudo mount -vt devpts devpts $(MOUNT_PT)/dev/pts -o gid=5,mode=620; \
> sudo mount -vt proc proc $(MOUNT_PT)/proc; \
> sudo mount -vt sysfs sysfs $(MOUNT_PT)/sys; \
> sudo mount -vt tmpfs tmpfs $(MOUNT_PT)/run; \
> if [ -h $(MOUNT_PT)/dev/shm ]; then :; \
>   sudo mkdir -pv $(MOUNT_PT)/$$(readlink $(MOUNT_PT)/dev/shm); \
> fi;

Yeah, I can see that I have not been mounting /dev/shm when building
these packages in chroot (before these, I always boot the
LFS+essentials (git, nfs, etc) build to check it boots, and to
recalculate the SBU.
> 
> #Since on debian, /dev/shm is not a symlink, the last mkdir is not executed.
> 
> #Note that a tmpfs is mounted on /run.
> 
> # Then
> /usr/sbin/chroot $(MOUNT_PT) \
>  /usr/bin/env -i \
>  HOME=/root TERM="$$TERM" \
>  PS1='(lfs chroot) \u:\w\$$ ' \
>  PATH=/bin:/usr/bin:/sbin:/usr/sbin \
> /bin/bash --login
> (again Makefile syntax).
> 
> Do I understand correctly that you and xry111 suggest that a tmpfs be mounted
> on /dev/shm? Without any perm change? Let me try...

No, I'm following Bruce's suggestion to just mount a tmpfs on /run,
as root *inside_chroot*.

> Yesss:
> root [ / ]# ls -al /dev/shm
> total 0
> drwxrwxrwt  2 root root   40 Mar 26 15:59 .
> drwxr-xr-x 21 root root 3900 Mar 26 09:48 ..
> 
> I just added a line:
> sudo mount -vt tmpfs tmpfs $(MOUNT_PT)/dev/shm; \
> 
> while still on host
> 
> Pierre

Interesting, there are clearly multiple workarounds.

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Xi Ruoyao via blfs-dev
On 2020-03-26 16:06 +0100, Pierre Labastie via blfs-dev wrote:
> Le 26/03/2020 à 14:41, Ken Moffat via blfs-dev a écrit :
> > On Thu, Mar 26, 2020 at 11:40:59AM +0100, Pierre Labastie via
> > blfs-dev wrote:
> > > Le 23/03/2020 à 03:49, Bruce Dubbs via blfs-dev a écrit :
> > > > On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
> > > > > On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> > > > > > On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via
> > > > > > blfs-dev wrote:
> > > > > > > On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> > > > > > > 
> > > > > > > >   # mount --bind /run /mnt/lfs/run
> > > > > > > 
> > > > > > > I think it's dangerous: potentially harmful to the host. 
> > > > > > > Some service running in the LFS chroot may overwrite the
> > > > > > > runtime directory of the service running on the host.
> > > > > > 
> > > > > > So, you are saying that packages from mozilla should not now
> > > > > > be built in chroot ?
> > > > > 
> > > > > No.  I think we need a better way.
> > > > > 
> > > > > > But, what do you mean by a service running in chroot ?  I
> > > > > > assume we are specifically talking of systemd here ?  Do
> > > > > > services not get started during the boot process ?  The
> > > > > > systemd instance in chroot has never started, so I assume it
> > > > > > will be ineffective and systemd on the host will only care
> > > > > > about services described in /etc/systemd ?
> > > > > 
> > > > > Maybe that's not a issue.  But still, /run contains lots of
> > > > > sockets of running services.  That means now we can connect
> > > > > host services from the chroot environment.  Even if it's not
> > > > > dangerous to host, it's polluting the new LFS system.
> > > > > 
> > > > > Consider /run/initctl.  We don't want something in chroot to
> > > > > switch the *host* to runlevel 1 :).
> > > > 
> > > > What about
> > > > 
> > > >  mount -t tmpfs /run
> > > > 
> > > > from within chroot?
> > > > 
> > > >   -- Bruce
> > > 
> > > FWIIW, here are my findings: On a debian host, we have:
> > > 
> > > lrwxrwxrwx 1 root root 8 mars  26 09:48 /run/shm -> /dev/shm
> > > drwxrwxrwt 2 root root 40 mars  26 11:02 /dev/shm
> > > 
> > > But when mounting /dev to $LFS with mount --bind, we have
> > > 
> > > drwxr-xr-x 2 root root 40 mars  26 11:01 /mnt/lfs/dev/shm
> > > 
> > > that is, the sticky bit and the write perm are lost for an average
> > > user.
> > > 
> > > Now trying to build FF in chroot (creating kernel virtual
> > > filesystems and chrooting with the lfs book instructions): - as
> > > root: needs to issue "export SHELL", but otherwise succeeds
> > > without anything else (/run exists as a tmpfs , but /run/shm does
> > > not exist) - as user: no need to run "export SHELL" (I guess "su -
> > > " does the export), but the perms to /dev/shm need to be
> > > changed to rwt
> > > 
> > > In both cases, the build goes to completion (not tried to install)
> > > after the indicated changes.
> > > 
> > > Another check I've done: on a lfs system (9.1-rc1 Sys V) as per
> > > the book, /dev/shm is drwxrwxrwt, and /run/shm does not exist. But
> > > everybody knows it is possible to build firefox Actually, if you
> > > just mount a devtmpfs:
> > > 
> > > mkdir tempdir-for-trying-something sudo mount -t devtmpfs devtmpfs
> > > tempdir-for-trying-something ls -ld
> > > tempdir-for-trying-something/shm
> > > 
> > > then the shm perms are drwxrwxrwt.
> > > 
> > > So I'd suggest our instructions be changed to mount a devtmpfs in
> > > $LFS/dev
> > > 
> > > But what makes me think I do something different from ĸen is that
> > > I never see: OSError: [Errno 38] Function not implemented
> > > 
> > > 
> > > Pierre
> > > 
> > I'll reply in a while with what I have now tested (after I've
> > assembled my notes).  I've gone with Bruce's suggestion to (only)
> > mount a tmpfs on /run when building in chroot - I think xry111 has
> > identified the issue (bind mounts are not recursive).
> > 
> > Meanwhile, if you do not see that OSError when building any of firefox,
> > mozjs{60,68}, seamonkey, thunderbird *in chroot* then you are doing
> > something different.  This issue was raised in
> > http://lists.linuxfromscratch.org/pipermail/blfs-support/2020-March/081897.html
> 
> The error reported by the OP ("This platform lacks a functioning sem_open
> implementation...") is the one I see when /dev/shm is not world writable, and
> the build is done as a regular user.
> When building as root, the error is different (something about an unknown
> shell).
> 
> Those are the only errors I see. Note that chroot is entered with the
> following commands (Makefile syntax):
>   sudo mount -v --bind /dev $(MOUNT_PT)/dev; \
> sudo mount -vt devpts devpts $(MOUNT_PT)/dev/pts -o gid=5,mode=620; \
> sudo mount -vt proc proc $(MOUNT_PT)/proc; \
> sudo mount -vt sysfs sysfs $(MOUNT_PT)/sys; \
> sudo mount -vt tmpfs tmpfs $(MOUNT_PT)/run; \
> if [ -h $(MOUNT_PT)/dev/shm 

Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 26/03/2020 à 14:41, Ken Moffat via blfs-dev a écrit :
> On Thu, Mar 26, 2020 at 11:40:59AM +0100, Pierre Labastie via
> blfs-dev wrote:
>> Le 23/03/2020 à 03:49, Bruce Dubbs via blfs-dev a écrit :
>>> On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
 On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via
> blfs-dev wrote:
>> On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
>>
>>>   # mount --bind /run /mnt/lfs/run
>>
>> I think it's dangerous: potentially harmful to the host. 
>> Some service running in the LFS chroot may overwrite the
>> runtime directory of the service running on the host.
>
> So, you are saying that packages from mozilla should not now
> be built in chroot ?

 No.  I think we need a better way.

> But, what do you mean by a service running in chroot ?  I
> assume we are specifically talking of systemd here ?  Do
> services not get started during the boot process ?  The
> systemd instance in chroot has never started, so I assume it
> will be ineffective and systemd on the host will only care
> about services described in /etc/systemd ?

 Maybe that's not a issue.  But still, /run contains lots of
 sockets of running services.  That means now we can connect
 host services from the chroot environment.  Even if it's not
 dangerous to host, it's polluting the new LFS system.

 Consider /run/initctl.  We don't want something in chroot to
 switch the *host* to runlevel 1 :).
>>>
>>> What about
>>>
>>>  mount -t tmpfs /run
>>>
>>> from within chroot?
>>>
>>>   -- Bruce
>>
>> FWIIW, here are my findings: On a debian host, we have:
>>
>> lrwxrwxrwx 1 root root 8 mars  26 09:48 /run/shm -> /dev/shm
>> drwxrwxrwt 2 root root 40 mars  26 11:02 /dev/shm
>>
>> But when mounting /dev to $LFS with mount --bind, we have
>>
>> drwxr-xr-x 2 root root 40 mars  26 11:01 /mnt/lfs/dev/shm
>>
>> that is, the sticky bit and the write perm are lost for an average
>> user.
>>
>> Now trying to build FF in chroot (creating kernel virtual
>> filesystems and chrooting with the lfs book instructions): - as
>> root: needs to issue "export SHELL", but otherwise succeeds
>> without anything else (/run exists as a tmpfs , but /run/shm does
>> not exist) - as user: no need to run "export SHELL" (I guess "su -
>> " does the export), but the perms to /dev/shm need to be
>> changed to rwt
>>
>> In both cases, the build goes to completion (not tried to install)
>> after the indicated changes.
>>
>> Another check I've done: on a lfs system (9.1-rc1 Sys V) as per
>> the book, /dev/shm is drwxrwxrwt, and /run/shm does not exist. But
>> everybody knows it is possible to build firefox Actually, if you
>> just mount a devtmpfs:
>>
>> mkdir tempdir-for-trying-something sudo mount -t devtmpfs devtmpfs
>> tempdir-for-trying-something ls -ld
>> tempdir-for-trying-something/shm
>>
>> then the shm perms are drwxrwxrwt.
>>
>> So I'd suggest our instructions be changed to mount a devtmpfs in
>> $LFS/dev
>>
>> But what makes me think I do something different from ĸen is that
>> I never see: OSError: [Errno 38] Function not implemented
>>
>>
>> Pierre
>>
> I'll reply in a while with what I have now tested (after I've
> assembled my notes).  I've gone with Bruce's suggestion to (only)
> mount a tmpfs on /run when building in chroot - I think xry111 has
> identified the issue (bind mounts are not recursive).
> 
> Meanwhile, if you do not see that OSError when building any of firefox,
> mozjs{60,68}, seamonkey, thunderbird *in chroot* then you are doing
> something different.  This issue was raised in
> http://lists.linuxfromscratch.org/pipermail/blfs-support/2020-March/081897.html

The error reported by the OP ("This platform lacks a functioning sem_open
implementation...") is the one I see when /dev/shm is not world writable, and
the build is done as a regular user.
When building as root, the error is different (something about an unknown 
shell).

Those are the only errors I see. Note that chroot is entered with the
following commands (Makefile syntax):
sudo mount -v --bind /dev $(MOUNT_PT)/dev; \
sudo mount -vt devpts devpts $(MOUNT_PT)/dev/pts -o gid=5,mode=620; \
sudo mount -vt proc proc $(MOUNT_PT)/proc; \
sudo mount -vt sysfs sysfs $(MOUNT_PT)/sys; \
sudo mount -vt tmpfs tmpfs $(MOUNT_PT)/run; \
if [ -h $(MOUNT_PT)/dev/shm ]; then :; \
  sudo mkdir -pv $(MOUNT_PT)/$$(readlink $(MOUNT_PT)/dev/shm); \
fi;

#Since on debian, /dev/shm is not a symlink, the last mkdir is not executed.

#Note that a tmpfs is mounted on /run.

# Then
/usr/sbin/chroot $(MOUNT_PT) \
 /usr/bin/env -i \
 HOME=/root TERM="$$TERM" \
 PS1='(lfs chroot) \u:\w\$$ ' \
 PATH=/bin:/usr/bin:/sbin:/usr/sbin \
/bin/bash --login
(again Makefile s

Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Ken Moffat via blfs-dev
On Thu, Mar 26, 2020 at 11:40:59AM +0100, Pierre Labastie via
blfs-dev wrote:
> Le 23/03/2020 à 03:49, Bruce Dubbs via blfs-dev a écrit :
> > On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
> >> On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> >>> On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via
> >>> blfs-dev wrote:
>  On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> 
> >   # mount --bind /run /mnt/lfs/run
> 
>  I think it's dangerous: potentially harmful to the host. 
>  Some service running in the LFS chroot may overwrite the
>  runtime directory of the service running on the host.
> >>>
> >>> So, you are saying that packages from mozilla should not now
> >>> be built in chroot ?
> >>
> >> No.  I think we need a better way.
> >>
> >>> But, what do you mean by a service running in chroot ?  I
> >>> assume we are specifically talking of systemd here ?  Do
> >>> services not get started during the boot process ?  The
> >>> systemd instance in chroot has never started, so I assume it
> >>> will be ineffective and systemd on the host will only care
> >>> about services described in /etc/systemd ?
> >>
> >> Maybe that's not a issue.  But still, /run contains lots of
> >> sockets of running services.  That means now we can connect
> >> host services from the chroot environment.  Even if it's not
> >> dangerous to host, it's polluting the new LFS system.
> >>
> >> Consider /run/initctl.  We don't want something in chroot to
> >> switch the *host* to runlevel 1 :).
> > 
> > What about
> > 
> >  mount -t tmpfs /run
> > 
> > from within chroot?
> > 
> >   -- Bruce
> 
> FWIIW, here are my findings: On a debian host, we have:
> 
> lrwxrwxrwx 1 root root 8 mars  26 09:48 /run/shm -> /dev/shm
> drwxrwxrwt 2 root root 40 mars  26 11:02 /dev/shm
> 
> But when mounting /dev to $LFS with mount --bind, we have
> 
> drwxr-xr-x 2 root root 40 mars  26 11:01 /mnt/lfs/dev/shm
> 
> that is, the sticky bit and the write perm are lost for an average
> user.
> 
> Now trying to build FF in chroot (creating kernel virtual
> filesystems and chrooting with the lfs book instructions): - as
> root: needs to issue "export SHELL", but otherwise succeeds
> without anything else (/run exists as a tmpfs , but /run/shm does
> not exist) - as user: no need to run "export SHELL" (I guess "su -
> " does the export), but the perms to /dev/shm need to be
> changed to rwt
> 
> In both cases, the build goes to completion (not tried to install)
> after the indicated changes.
> 
> Another check I've done: on a lfs system (9.1-rc1 Sys V) as per
> the book, /dev/shm is drwxrwxrwt, and /run/shm does not exist. But
> everybody knows it is possible to build firefox Actually, if you
> just mount a devtmpfs:
> 
> mkdir tempdir-for-trying-something sudo mount -t devtmpfs devtmpfs
> tempdir-for-trying-something ls -ld
> tempdir-for-trying-something/shm
> 
> then the shm perms are drwxrwxrwt.
> 
> So I'd suggest our instructions be changed to mount a devtmpfs in
> $LFS/dev
> 
> But what makes me think I do something different from ĸen is that
> I never see: OSError: [Errno 38] Function not implemented
> 
> 
> Pierre
> 
I'll reply in a while with what I have now tested (after I've
assembled my notes).  I've gone with Bruce's suggestion to (only)
mount a tmpfs on /run when building in chroot - I think xry111 has
identified the issue (bind mounts are not recursive).

Meanwhile, if you do not see that OSError when building any of firefox,
mozjs{60,68}, seamonkey, thunderbird *in chroot* then you are doing
something different.  This issue was raised in
http://lists.linuxfromscratch.org/pipermail/blfs-support/2020-March/081897.html

I'll document *how* I tested so that people can pick it apart if
they wish :)

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Xi Ruoyao via blfs-dev
On 2020-03-26 09:41 +0100, Pierre Labastie via blfs-dev wrote:
> So my suspicion is that when we mount --bind /dev, we lose some permissions,
> and those permissions are needed for FF. I'll have to try again without
> /run/shm

The problem is /dev/shm is a seperated tmpfs, instead of a normal directory in
/dev.  mount --bind is not recursive so in the bind mount /dev/shm is a normal
directory, which is not writable.

I suggest to simply mount a tmpfs on /dev/shm:

mount --bind /dev $LFS/dev
mount -v -t tmpfs tmpfs $LFS/dev/shm

> Will report
> Pierre
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 23/03/2020 à 03:49, Bruce Dubbs via blfs-dev a écrit :
> On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
>> On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
>>> On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:
 On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:

>   # mount --bind /run /mnt/lfs/run

 I think it's dangerous: potentially harmful to the host.  Some service
 running
 in the LFS chroot may overwrite the runtime directory of the service 
 running
 on
 the host.
>>>
>>> So, you are saying that packages from mozilla should not now be
>>> built in chroot ?
>>
>> No.  I think we need a better way.
>>
>>> But, what do you mean by a service running in chroot ?  I assume we
>>> are specifically talking of systemd here ?  Do services not get
>>> started during the boot process ?  The systemd instance in chroot
>>> has never started, so I assume it will be ineffective and systemd on
>>> the host will only care about services described in /etc/systemd ?
>>
>> Maybe that's not a issue.  But still, /run contains lots of sockets of 
>> running
>> services.  That means now we can connect host services from the chroot
>> environment.  Even if it's not dangerous to host, it's polluting the new LFS
>> system.
>>
>> Consider /run/initctl.  We don't want something in chroot to switch the 
>> *host*
>> to runlevel 1 :).
> 
> What about
> 
>  mount -t tmpfs /run
> 
> from within chroot?
> 
>   -- Bruce

FWIIW, here are my findings:
On a debian host, we have:

lrwxrwxrwx 1 root root 8 mars  26 09:48 /run/shm -> /dev/shm
drwxrwxrwt 2 root root 40 mars  26 11:02 /dev/shm

But when mounting /dev to $LFS with mount --bind, we have

drwxr-xr-x 2 root root 40 mars  26 11:01 /mnt/lfs/dev/shm

that is, the sticky bit and the write perm are lost for an average user.

Now trying to build FF in chroot (creating kernel virtual filesystems and
chrooting with the lfs book instructions):
- as root: needs to issue "export SHELL", but otherwise succeeds without
  anything else (/run exists as a tmpfs , but /run/shm does not exist)
- as user: no need to run "export SHELL" (I guess "su - " does the
  export), but the perms to /dev/shm need to be changed to rwt

In both cases, the build goes to completion (not tried to install) after the
indicated changes.

Another check I've done: on a lfs system (9.1-rc1 Sys V) as per the book,
/dev/shm is drwxrwxrwt, and /run/shm does not exist. But everybody knows it is
possible to build firefox Actually, if you just mount a devtmpfs:

mkdir tempdir-for-trying-something
sudo mount -t devtmpfs devtmpfs tempdir-for-trying-something
ls -ld tempdir-for-trying-something/shm

then the shm perms are drwxrwxrwt.

So I'd suggest our instructions be changed to mount a devtmpfs in $LFS/dev

But what makes me think I do something different from ĸen is that I never see:
OSError: [Errno 38] Function not implemented


Pierre

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-26 Thread Pierre Labastie via blfs-dev
Le 23/03/2020 à 15:29, Ken Moffat via blfs-dev a écrit :
> On Sun, Mar 22, 2020 at 09:49:02PM -0500, Bruce Dubbs via blfs-dev wrote:
>> On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
>>> On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
 On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:
> On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:

 So, you are saying that packages from mozilla should not now be
 built in chroot ?
>>>
>>> No.  I think we need a better way.
>>>
 But, what do you mean by a service running in chroot ?  I assume we
 are specifically talking of systemd here ?  Do services not get
 started during the boot process ?  The systemd instance in chroot
 has never started, so I assume it will be ineffective and systemd on
 the host will only care about services described in /etc/systemd ?
>>>
>>> Maybe that's not a issue.  But still, /run contains lots of sockets of 
>>> running
>>> services.  That means now we can connect host services from the chroot
>>> environment.  Even if it's not dangerous to host, it's polluting the new LFS
>>> system.
>>>
>>> Consider /run/initctl.  We don't want something in chroot to switch the 
>>> *host*
>>> to runlevel 1 :).
>>
> 
> As a developer, I take the view that I'll do what I want with my
> machine, including shooting myself in the foot :)  For the book,
> yes, I take your point.
> 
>> What about
>>
>>  mount -t tmpfs /run
>>
>> from within chroot?
>>
>>   -- Bruce
> 
> I suspect that what is actually needed is /run/shm.  Will need to do
> some testing and stracing. ...
> 

FWIIW, I've succesfully built FF with these two instructions added to what is
in lfs on the "Preparing Virtual Kernel File Systems" page:
ln -s /dev/shm /run
chmod a+wt /dev/shm

Note that:
The host is debian. Debian has:
drwxrwxrwt 2 root root 40 mars  26 08:48 /dev/shm
and
lrwxrwxrwx 1 root root 8 mars  23 21:29 /run/shm -> /dev/shm

but when /dev is (mount --bind)ed, /dev/shm becomes
drwxr-xr-x 21 root root 3900 mars  23 21:29 /mnt/lfs/dev

The change in permission on /dev/shm _is_ needed to allow the configuration to
run.

Note that I have not tried to just change the permission on /dev/shm and not
symlinking /run/shm to /dev.

Ah, I just launched a VM with lfs (9.1-rc1), and /dev/shm is exactly the same
as above (a directory with rwt permissions for all). And there is no /run/shm...

So my suspicion is that when we mount --bind /dev, we lose some permissions,
and those permissions are needed for FF. I'll have to try again without /run/shm

Will report
Pierre
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-23 Thread Ken Moffat via blfs-dev
On Sun, Mar 22, 2020 at 09:49:02PM -0500, Bruce Dubbs via blfs-dev wrote:
> On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:
> > On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> > > On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:
> > > > On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> > > 
> > > So, you are saying that packages from mozilla should not now be
> > > built in chroot ?
> > 
> > No.  I think we need a better way.
> > 
> > > But, what do you mean by a service running in chroot ?  I assume we
> > > are specifically talking of systemd here ?  Do services not get
> > > started during the boot process ?  The systemd instance in chroot
> > > has never started, so I assume it will be ineffective and systemd on
> > > the host will only care about services described in /etc/systemd ?
> > 
> > Maybe that's not a issue.  But still, /run contains lots of sockets of 
> > running
> > services.  That means now we can connect host services from the chroot
> > environment.  Even if it's not dangerous to host, it's polluting the new LFS
> > system.
> > 
> > Consider /run/initctl.  We don't want something in chroot to switch the 
> > *host*
> > to runlevel 1 :).
> 

As a developer, I take the view that I'll do what I want with my
machine, including shooting myself in the foot :)  For the book,
yes, I take your point.

> What about
> 
>  mount -t tmpfs /run
> 
> from within chroot?
> 
>   -- Bruce

I suspect that what is actually needed is /run/shm.  Will need to do
some testing and stracing. ...

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-22 Thread Bruce Dubbs via blfs-dev

On 3/22/20 9:25 PM, Xi Ruoyao via blfs-dev wrote:

On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:

On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:

On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:


  # mount --bind /run /mnt/lfs/run


I think it's dangerous: potentially harmful to the host.  Some service
running
in the LFS chroot may overwrite the runtime directory of the service running
on
the host.


So, you are saying that packages from mozilla should not now be
built in chroot ?


No.  I think we need a better way.


But, what do you mean by a service running in chroot ?  I assume we
are specifically talking of systemd here ?  Do services not get
started during the boot process ?  The systemd instance in chroot
has never started, so I assume it will be ineffective and systemd on
the host will only care about services described in /etc/systemd ?


Maybe that's not a issue.  But still, /run contains lots of sockets of running
services.  That means now we can connect host services from the chroot
environment.  Even if it's not dangerous to host, it's polluting the new LFS
system.

Consider /run/initctl.  We don't want something in chroot to switch the *host*
to runlevel 1 :).


What about

 mount -t tmpfs /run

from within chroot?

  -- Bruce
--
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-22 Thread Xi Ruoyao via blfs-dev
On 2020-03-23 01:44 +, Ken Moffat via blfs-dev wrote:
> On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:
> > On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> > 
> > >  # mount --bind /run /mnt/lfs/run
> > 
> > I think it's dangerous: potentially harmful to the host.  Some service
> > running
> > in the LFS chroot may overwrite the runtime directory of the service running
> > on
> > the host.
> 
> So, you are saying that packages from mozilla should not now be
> built in chroot ?

No.  I think we need a better way.

> But, what do you mean by a service running in chroot ?  I assume we
> are specifically talking of systemd here ?  Do services not get
> started during the boot process ?  The systemd instance in chroot
> has never started, so I assume it will be ineffective and systemd on
> the host will only care about services described in /etc/systemd ?

Maybe that's not a issue.  But still, /run contains lots of sockets of running
services.  That means now we can connect host services from the chroot
environment.  Even if it's not dangerous to host, it's polluting the new LFS
system.

Consider /run/initctl.  We don't want something in chroot to switch the *host*
to runlevel 1 :).
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-22 Thread Ken Moffat via blfs-dev
On Mon, Mar 23, 2020 at 09:12:18AM +0800, Xi Ruoyao via blfs-dev wrote:
> On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:
> 
> >  # mount --bind /run /mnt/lfs/run
> 
> I think it's dangerous: potentially harmful to the host.  Some service running
> in the LFS chroot may overwrite the runtime directory of the service running 
> on
> the host.

So, you are saying that packages from mozilla should not now be
built in chroot ?

But, what do you mean by a service running in chroot ?  I assume we
are specifically talking of systemd here ?  Do services not get
started during the boot process ?  The systemd instance in chroot
has never started, so I assume it will be ineffective and systemd on
the host will only care about services described in /etc/systemd ?

On a sysvinit system I have not encountered a problem, but I have no
recent experience of systemd.

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


Re: [blfs-dev] Python multiprocessing checks in chroot

2020-03-22 Thread Xi Ruoyao via blfs-dev
On 2020-03-22 21:34 +, Ken Moffat via blfs-dev wrote:

>  # mount --bind /run /mnt/lfs/run

I think it's dangerous: potentially harmful to the host.  Some service running
in the LFS chroot may overwrite the runtime directory of the service running on
the host.
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page


[blfs-dev] Python multiprocessing checks in chroot

2020-03-22 Thread Ken Moffat via blfs-dev
This weekend we had a problem where someone was trying to build
mozjs60 in chroot and failed.  I've seen this myself on some of the
rare occasions I've built desktops in chroot.

Key facts:

1. The solution is (as root, on the *host* system) bind /mnt/lfs/run
to /run, i.e.
 # mount --bind /run /mnt/lfs/run

2. Where this is required, the error messages will say
OSError: [Errno 38] Function not implemented
and a few lines above that will be a mention of multiprocessing.

Usually I don't build the desktop in chroot, but maybe a few tiems a
year I have found that appropriate for testing.  On my current
laptop this also seems to be the easiest approach (at least until
xfce and network-manager-applet are in place).

I first saw this when building firefox 53 in 2017, and I'd assumed
it was covered somewhere in one of the books - but now, I don't
think it is mentioned.  If that is so, I guess it would be useful to
add an extra copy member for it, to include in a Note for the
affected packages.

Something like (using '*' to show things to be emphasised)

"If you are building in chroot, on the *host* system you must ensure
/mnt/lfs/run in LFS is bind-mounted, otherwise the configuration
will report

 OSError: [Errno 38] Function not implemented

and a few lines above that it will mention the python
multiprocessing package.

Work around this by, *on the host system*, as root, executing:
mount --bind /run /mnt/lfs/run "

Does that sound ok ?  Will it need any massaging to stop jhalfs
trying to run it as a matter of course ?

If ok, I think the following need it:

mozjs60
mozjs68 when it goes in (unless the python devs have completely
changed how their multiprocessing module tests the system)
node-js
firefox
seamonkey
thunderbird

I can do this, but I'm not sure if I've maybe overlooked something
that already covers this ?

ĸen
-- 
When alle is ſayed and all is done, ye must chooſe your faces wisely,
for soon enouff ye will be playing with fyre."
  The Nice and Accurate Prophecies of Agnes Nutter, Prophecy 5004
-- 
http://lists.linuxfromscratch.org/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page