[gentoo-user] an efficient idea for an alternative portage synchronisation

[caveman رَجُلُ الْكَهْفِ 穴居人]

tl;dr - i'm suggesting a new file syncing protocol
for portage syncing.  details of this one is in
section 2.


1. background
-
rsync needs to read all files in order to compare
them.  this is too expensive and doesn't scale as
portage's tree grows in size..

on the other hand, git gets away with this, by
maintaining a history of edits.  so git doesn't
need to compare all files, instead it walks
through the history.

but git has another issue:  the history getting
too big.  this causes:
- `git clone` to needlessly take too long, as
  many old histories become irrelevant as they
  get fully overwridden by newer ones.
- this also causes `git pull` to be slower
  than needed, as the history is not ideally
  compressed.
- plus, the disk space that's wasted for
  histories.


2. new protocol
---
to solve issues above, i think the ideal solution
is this protocol:
- each history is a number representing a
  logical clock.  1st history is 0, 2nd is 1,
  etc.
- the server maintains a list of N past many
  histories of the portage tree.
- when a client requests to update its portage
  tree, it tells the server its current
  history.  e.g. say client is currently
  located in logical time 1234567.
- the server is maintaining only the past N
  histories:
- if 1234567 is behind those maintained N
  ones, then the server sends a full
  portage tree from scratch.
- if 1234567 is within those maintained N
  ones, then the server has two options:
(1) either send all changes since
1234567, as they happened
historically.  this is a bad idea.
no good reason for it.

(2) better: the server can send the
compressed histories.  compressed
histories are done once, and
cached, in a scalable way.  the
cache itself is incremental, so
updating the cache is cheap
(details section 2.2.).

e.g. if there are 5000 histories
that the client lacks since time
1234567, then there is a chance
that many of the changes are just
a waste of time.  e.g. add a file,
then delete the same file, then
add a different file again.  so
why not just lie about the
history, and send the last file,
escaping ones int he middle?  same
can be thought about diffs to code
blocks.

2.1. properties of this new protocol

so this new protocol has these properties:
- unlike rsync, it doesn't need to compare all files
  individually.
- unlike git, the history doesn't grow on the
  client.  history remains only a single
  number representing a logical clock.
- the history on the server is limited to N
  past entries.  no devs will cry, because
  this is not a code collaboration app, but
  simply a file synchronisation app to replace
  rsync.  so the admins are free to set N as
  small as they please, without worrying about
  harming collaborating devs.
- server has the option to compress histories
  to clients, and these histories are
  cacheable for more performance.


2.2. how it will feel to admins/devs

- the devs simply commit their changes to the
  portage tree via git.
- the git server will have hooks to execute an
  external command for this new protocol, that
  will calculate all diffs necessary in order
  to build a new history.

  e.g. if current history is 3, and a dev
  makes a new commit via git, then the git
  hooks will execute the external command to
  calculate the diff for the affected files by
  the git commit, such that history 30001 is
  created.

  the hooked external command will also see if
  it can compress the histories, for the past
  M many entries since 30001.

  so that clients that live in time 30001-M,
  who ask for 30001, can get the compressed
  history instead of raw actual histories from
  30001-m to 30001.

ty,
cm.

Re: [gentoo-user] Python-3.9 and emerge problems

[John Covici]

On Fri, 18 Jun 2021 10:00:17 -0400,
Neil Bothwick wrote:
> 
> [1  ]
> On Fri, 18 Jun 2021 09:54:09 -0400, John Covici wrote:
> 
> > > Same problem with 3.0.20-r3 but then I found
> > > https://bugs.gentoo.org/796584, changing sets.conf got rid of the
> > > issue.  
> > 
> > hmmm, I don't have a file like that at all, what should be in that
> > file -- I don't even have a /usr/portage at all.
> > 
> The system sets are in /usr/share/portage/config/sets, but they all had
> exclude-files where needed. The culprits were my user-defined sets in
> /etc/sets.conf. For example:
> 
> [kernels]
> class = portage.sets.dbapi.OwnerSet
> world-candidate = False
> files = /usr/src
> exclude-files = ''
> 
> Adding that last line stopped the breakage. This sort of change should
> have been accompanied by an einfo message at the very least, preferable a
> news item.

Thanks much -- that seems to have fixed it.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici wb2una
 cov...@ccs.covici.com

Re: [gentoo-user] Python-3.9 and emerge problems

[Jacques Montier]

* *


Le ven. 18 juin 2021 à 16:00, Neil Bothwick  a écrit :

> On Fri, 18 Jun 2021 09:54:09 -0400, John Covici wrote:
>
> > > Same problem with 3.0.20-r3 but then I found
> > > https://bugs.gentoo.org/796584, changing sets.conf got rid of the
> > > issue.
> >
> > hmmm, I don't have a file like that at all, what should be in that
> > file -- I don't even have a /usr/portage at all.
> >
> The system sets are in /usr/share/portage/config/sets, but they all had
> exclude-files where needed. The culprits were my user-defined sets in
> /etc/sets.conf. For example:
>
> [kernels]
> class = portage.sets.dbapi.OwnerSet
> world-candidate = False
> files = /usr/src
> exclude-files = ''
>
> Adding that last line stopped the breakage. This sort of change should
> have been accompanied by an einfo message at the very least, preferable a
> news item.
>
>
> --
> Neil Bothwick
>
> Top Oxymorons Number 43: Genuine imitation
>


Hello again,

Yes Neil, that did the trick ! 👍
Thanks a lot to all of you !

--
Jacques

Re: [gentoo-user] an efficient idea for an alternative portage synchronisation

[Michael Jones]

On Fri, Jun 18, 2021, 07:10 caveman رَجُلُ الْكَهْفِ 穴居人 <
toraboracave...@protonmail.com> wrote:

> tl;dr - i'm suggesting a new file syncing protocol
> for portage syncing.  details of this one is in
> section 2.
>
>
> 1. background
> -
> rsync needs to read all files in order to compare
> them.  this is too expensive and doesn't scale as
> portage's tree grows in size..
>
> on the other hand, git gets away with this, by
> maintaining a history of edits.  so git doesn't
> need to compare all files, instead it walks
> through the history.
>
> but git has another issue:  the history getting
> too big.  this causes:
> - `git clone` to needlessly take too long, as
>   many old histories become irrelevant as they
>   get fully overwridden by newer ones.
> - this also causes `git pull` to be slower
>   than needed, as the history is not ideally
>   compressed.
> - plus, the disk space that's wasted for
>   histories.
>
>
> 2. new protocol
> ---
> to solve issues above, i think the ideal solution
> is this protocol:
> - each history is a number representing a
>   logical clock.  1st history is 0, 2nd is 1,
>   etc.
> - the server maintains a list of N past many
>   histories of the portage tree.
> - when a client requests to update its portage
>   tree, it tells the server its current
>   history.  e.g. say client is currently
>   located in logical time 1234567.
> - the server is maintaining only the past N
>   histories:
> - if 1234567 is behind those maintained N
>   ones, then the server sends a full
>   portage tree from scratch.
> - if 1234567 is within those maintained N
>   ones, then the server has two options:
> (1) either send all changes since
> 1234567, as they happened
> historically.  this is a bad idea.
> no good reason for it.
>
> (2) better: the server can send the
> compressed histories.  compressed
> histories are done once, and
> cached, in a scalable way.  the
> cache itself is incremental, so
> updating the cache is cheap
> (details section 2.2.).
>
> e.g. if there are 5000 histories
> that the client lacks since time
> 1234567, then there is a chance
> that many of the changes are just
> a waste of time.  e.g. add a file,
> then delete the same file, then
> add a different file again.  so
> why not just lie about the
> history, and send the last file,
> escaping ones int he middle?  same
> can be thought about diffs to code
> blocks.
>
> 2.1. properties of this new protocol
> 
> so this new protocol has these properties:
> - unlike rsync, it doesn't need to compare all files
>   individually.
> - unlike git, the history doesn't grow on the
>   client.  history remains only a single
>   number representing a logical clock.
> - the history on the server is limited to N
>   past entries.  no devs will cry, because
>   this is not a code collaboration app, but
>   simply a file synchronisation app to replace
>   rsync.  so the admins are free to set N as
>   small as they please, without worrying about
>   harming collaborating devs.
> - server has the option to compress histories
>   to clients, and these histories are
>   cacheable for more performance.
>
>
> 2.2. how it will feel to admins/devs
> 
> - the devs simply commit their changes to the
>   portage tree via git.
> - the git server will have hooks to execute an
>   external command for this new protocol, that
>   will calculate all diffs necessary in order
>   to build a new history.
>
>   e.g. if current history is 3, and a dev
>   makes a new commit via git, then the git
>   hooks will execute the external command to
>   calculate the diff for the affected files by
>   the git commit, such that history 30001 is
>   created.
>
>   the hooked external command will also see if
>   it can compress the histories, for the past
>   M many entries since 30001.
>
>   so that clients that live in time 30001-M,
>   who ask for 30001, can get the compressed
>   history instead of raw actual histories from
>   30001-m to 30001.
>
> ty,
> cm
>


It seems like you are almost asking for git's --clone-depth and
--sync-depth flags.

Its not an exact match for your proposal but its very close.

>

Re: [gentoo-user] Python-3.9 and emerge problems

[Neil Bothwick]

On Fri, 18 Jun 2021 09:54:09 -0400, John Covici wrote:

> > Same problem with 3.0.20-r3 but then I found
> > https://bugs.gentoo.org/796584, changing sets.conf got rid of the
> > issue.  
> 
> hmmm, I don't have a file like that at all, what should be in that
> file -- I don't even have a /usr/portage at all.
> 
The system sets are in /usr/share/portage/config/sets, but they all had
exclude-files where needed. The culprits were my user-defined sets in
/etc/sets.conf. For example:

[kernels]
class = portage.sets.dbapi.OwnerSet
world-candidate = False
files = /usr/src
exclude-files = ''

Adding that last line stopped the breakage. This sort of change should
have been accompanied by an einfo message at the very least, preferable a
news item.


-- 
Neil Bothwick

Top Oxymorons Number 43: Genuine imitation


pgpiq1yd0KKBM.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Python-3.9 and emerge problems

[John Covici]

On Fri, 18 Jun 2021 09:11:29 -0400,
Neil Bothwick wrote:
> 
> [1  ]
> On Fri, 18 Jun 2021 13:07:10 +0100, Neil Bothwick wrote:
> 
> > I had the same a couple of days ago and had to downgrade portage from
> > 3.0.20 to 3.0.18, by untarring the binary package to /. However, that
> > needed python 3.8, so I also had to untar the binary package for that.
> > Then I re-emerged portage-3.0.18 and python-3.8 and masked
> > portage-3.0.20. Once I had re-emerged portage-3.0.18, it worked with
> > python 3.9.
> > 
> > I see that portage 3.0.20-r3 is here now, so I'll try emerging that on
> > one system and see what happens.
> 
> 
> Same problem with 3.0.20-r3 but then I found
> https://bugs.gentoo.org/796584, changing sets.conf got rid of the issue.

hmmm, I don't have a file like that at all, what should be in that
file -- I don't even have a /usr/portage at all.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici wb2una
 cov...@ccs.covici.com

Re: [gentoo-user] Python-3.9 and emerge problems

[John Covici]

On Fri, 18 Jun 2021 08:07:10 -0400,
Neil Bothwick wrote:
> 
> [1  ]
> On Fri, 18 Jun 2021 11:46:27 +0200, Jacques Montier wrote:
> 
> > #emerge --sync
> > #emerge --oneshot sys-apps/portage
> > #emerge -auvDN --with-bdeps=y --keep-going world
> > 
> > I get these errors :
> > 
> > Traceback (most recent call last):
> >   File "/usr/lib/python-exec/python3.9/emerge", line 51, in 
> > retval = emerge_main()
> >   File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in
> > emerge_main
> > return run_action(emerge_config)
> >   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line
> > 3392, in run_action
> > retval = action_build(emerge_config, spinner=spinner)
> >   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 354,
> > in action_build
> > success, mydepgraph, favorites = backtrack_depgraph(
> >   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> > 10005, in backtrack_depgraph
> > return _backtrack_depgraph(settings, trees, myopts, myparams,
> >   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> > 10043, in _backtrack_depgraph
> > success, favorites = mydepgraph.select_files(myfiles)
> >   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> > 4055, in select_files
> > return self._select_files(args)
> >   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> > 4189, in _select_files
> > set_atoms = root_config.setconfig.getSetAtoms(s)
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> > line 271, in getSetAtoms
> > myatoms.update(self.getSetAtoms(s,
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> > line 271, in getSetAtoms
> > myatoms.update(self.getSetAtoms(s,
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> > line 260, in getSetAtoms
> > myatoms = myset.getAtoms()
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> > 58, in getAtoms
> > self._load()
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> > 53, in _load
> > self.load()
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> > 111, in load
> > self._setAtoms(self.mapPathsToAtoms(self._files,
> >   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> > 83, in mapPathsToAtoms
> > for p in exclude_paths:
> > TypeError: 'NoneType' object is not iterable
> > 
> > And i can't do anything else...
> > 
> > I see that python-3.8 has gone away
> > # eselect python list
> > Available Python interpreters, in order of preference:
> >   [1]   python3.9
> 
> I has the same a couple of days ago and had to downgrade portage from
> 3.0.20 to 3.0.18, by untarring the binary package to /. However, that
> needed python 3.8, so I also had to untar the binary package for that.
> Then I re-emerged portage-3.0.18 and python-3.8 and masked
> portage-3.0.20. Once I had re-emerged portage-3.0.18, it worked with
> python 3.9.
> 
> I see that portage 3.0.20-r3 is here now, so I'll try emerging that on
> one system and see what happens.

No joy  on doing that here -- last time  I had to restore the whole
3.8 site-packages directory from before I did the emerge -- just
restoring 3.0.18 still gives me the traceback -- this is major
annoyance.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici wb2una
 cov...@ccs.covici.com

Re: [gentoo-user] Python-3.9 and emerge problems

[Neil Bothwick]

On Fri, 18 Jun 2021 13:07:10 +0100, Neil Bothwick wrote:

> I had the same a couple of days ago and had to downgrade portage from
> 3.0.20 to 3.0.18, by untarring the binary package to /. However, that
> needed python 3.8, so I also had to untar the binary package for that.
> Then I re-emerged portage-3.0.18 and python-3.8 and masked
> portage-3.0.20. Once I had re-emerged portage-3.0.18, it worked with
> python 3.9.
> 
> I see that portage 3.0.20-r3 is here now, so I'll try emerging that on
> one system and see what happens.


Same problem with 3.0.20-r3 but then I found
https://bugs.gentoo.org/796584, changing sets.conf got rid of the issue.


-- 
Neil Bothwick

Of all the people I've met you're certainly one of them.


pgpR5OBXzEKpF.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Disable password required to mount removable hard disk.

[William Kenworthy]

The password problem was solved back in April, but some more info on the
semi random disk assignments might help someone as the question keeps
popping up:

I use genkernel and grub to boot via MBR - however root is on a btrfs
raid 10 (all SSD's, 3 are whole disk and one has root on  partition 3
alongside boot and swap)

When using /dev/sdx notation, the grub hardware mapping (root)
semi-randomly moves between disks

OK, so I tried using UUID's - the same

So I tried using labels - still happens!!!

Interestingly, a suspend/resume always works as expected then a couple
of days ago I stumbled on a genkernel bug (#796272
) with module loading -
its a bit of a corner case but its looking like I might have found the
cause.

BillK



On 18/6/21 2:13 pm, Hund wrote:
> On April 1, 2021 10:12:00 AM GMT+02:00, William Kenworthy 
>  wrote:
>> I have used fstab in the past -its more a workaround that breaks (i.e, a
>> disk usually, but not always appears as /dev/sde [...]
> fstab? Workaround? Use UUID.
>
>
> --
> Hund
>

Re: [gentoo-user] New install - root is mounted read-only

[Neil Bothwick]

On Fri, 18 Jun 2021 10:55:32 +0100, Wols Lists wrote:

> I've started tackling my new build again, and when it boots root is
> read-only. Hopefully I've just missed something stupid, but how to I get
> it to transition read-write?
> 
> System is grub, systemd, and root is an lv ...
> 
> Do I need to do anything special with the initrd? Manually remounting
> fixes it fine as far as I can tell.

What are you using to build the initrd? What is in fstab? What kernel
options are you using, /proc/cmdline?


-- 
Neil Bothwick

Employ teenagers - while they know everything.


pgpTD7YzfFHaN.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Python-3.9 and emerge problems

[Neil Bothwick]

On Fri, 18 Jun 2021 11:46:27 +0200, Jacques Montier wrote:

> #emerge --sync
> #emerge --oneshot sys-apps/portage
> #emerge -auvDN --with-bdeps=y --keep-going world
> 
> I get these errors :
> 
> Traceback (most recent call last):
>   File "/usr/lib/python-exec/python3.9/emerge", line 51, in 
> retval = emerge_main()
>   File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in
> emerge_main
> return run_action(emerge_config)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line
> 3392, in run_action
> retval = action_build(emerge_config, spinner=spinner)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 354,
> in action_build
> success, mydepgraph, favorites = backtrack_depgraph(
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 10005, in backtrack_depgraph
> return _backtrack_depgraph(settings, trees, myopts, myparams,
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 10043, in _backtrack_depgraph
> success, favorites = mydepgraph.select_files(myfiles)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 4055, in select_files
> return self._select_files(args)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 4189, in _select_files
> set_atoms = root_config.setconfig.getSetAtoms(s)
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 260, in getSetAtoms
> myatoms = myset.getAtoms()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> 58, in getAtoms
> self._load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> 53, in _load
> self.load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> 111, in load
> self._setAtoms(self.mapPathsToAtoms(self._files,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> 83, in mapPathsToAtoms
> for p in exclude_paths:
> TypeError: 'NoneType' object is not iterable
> 
> And i can't do anything else...
> 
> I see that python-3.8 has gone away
> # eselect python list
> Available Python interpreters, in order of preference:
>   [1]   python3.9

I has the same a couple of days ago and had to downgrade portage from
3.0.20 to 3.0.18, by untarring the binary package to /. However, that
needed python 3.8, so I also had to untar the binary package for that.
Then I re-emerged portage-3.0.18 and python-3.8 and masked
portage-3.0.20. Once I had re-emerged portage-3.0.18, it worked with
python 3.9.

I see that portage 3.0.20-r3 is here now, so I'll try emerging that on
one system and see what happens.


-- 
Neil Bothwick

C Error #011: First C Program, huh?


pgp6z0ZnUTBns.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] New install - root is mounted read-only

[tastytea]

On 2021-06-18 10:55+0100 Wols Lists  wrote:

> I've started tackling my new build again, and when it boots root is
> read-only. Hopefully I've just missed something stupid, but how to I
> get it to transition read-write?
> 
> System is grub, systemd, and root is an lv ...
> 
> Do I need to do anything special with the initrd? Manually remounting
> fixes it fine as far as I can tell.

Do you use a custom initrd? The kernel usually mounts / ro at first and
the initrd remounts it with `mount -o remount,rw /` later.
You can also mount / rw from the beginning by specifying “rw” instead of
“ro” on the kernel commandline¹. The current kernel commandline is
visible in /proc/cmdline.

Kind regards, tastytea

¹ 

-- 
Get my PGP key with `gpg --locate-keys tasty...@tastytea.de` or at
.


pgpViPptvdmV1.pgp
Description: Digitale Signatur von OpenPGP

Re: [gentoo-user] Python-3.9 and emerge problems

[John Covici]

On Fri, 18 Jun 2021 05:46:27 -0400,
Jacques Montier wrote:
> 
> [1  ]
> [1.1  ]
> Hello all,
> 
> This morning :
> 
> #emerge --sync
> #emerge --oneshot sys-apps/portage
> #emerge -auvDN --with-bdeps=y --keep-going world
> 
> I get these errors :
> 
> Traceback (most recent call last):
>   File "/usr/lib/python-exec/python3.9/emerge", line 51, in 
> retval = emerge_main()
>   File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in
> emerge_main
> return run_action(emerge_config)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 3392, in
> run_action
> retval = action_build(emerge_config, spinner=spinner)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 354, in
> action_build
> success, mydepgraph, favorites = backtrack_depgraph(
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 10005,
> in backtrack_depgraph
> return _backtrack_depgraph(settings, trees, myopts, myparams,
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 10043,
> in _backtrack_depgraph
> success, favorites = mydepgraph.select_files(myfiles)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 4055,
> in select_files
> return self._select_files(args)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 4189,
> in _select_files
> set_atoms = root_config.setconfig.getSetAtoms(s)
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
> 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
> 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
> 260, in getSetAtoms
> myatoms = myset.getAtoms()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line 58,
> in getAtoms
> self._load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line 53,
> in _load
> self.load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line 111,
> in load
> self._setAtoms(self.mapPathsToAtoms(self._files,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line 83,
> in mapPathsToAtoms
> for p in exclude_paths:
> TypeError: 'NoneType' object is not iterable
> 
> And i can't do anything else...
> 
> I see that python-3.8 has gone away
> # eselect python list
> Available Python interpreters, in order of preference:
>   [1]   python3.9
> 
> emerge --info file attached
> 
> I don't really know what to do.
> Any idea ?
> 
> Thanks a lot in advance,

...

I got a similar error -- I have a strange idea  about this -- see if
you have any instances ofthread.isAlive( in portage or anywhere else
in your python3.9 -- if you do you need to change them to
thread.is_alive( ... python changed the name and I don't think some
people got the message -- apparently it has been depricated for quite
a while now.

Just a wild guess.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici wb2una
 cov...@ccs.covici.com

Re: [gentoo-user] Python-3.9 and emerge problems

[Wols Lists]

On 18/06/21 10:46, Jacques Montier wrote:
> Hello all,
> 
> This morning :
> 
> #emerge --sync
> #emerge --oneshot sys-apps/portage
> #emerge -auvDN --with-bdeps=y --keep-going world

You can't try just updating python? When I tried to emerge portage it
blew up with loads of stuff about 3.8 and 3.9, so I just did an "emerge
-uDN python" and everything started working (well, emerge at least ... :-)

Cheers,
Wol
> 
> I get these errors :
> 
> Traceback (most recent call last):
>   File "/usr/lib/python-exec/python3.9/emerge", line 51, in 
> retval = emerge_main()
>   File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in
> emerge_main
> return run_action(emerge_config)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 3392,
> in run_action
> retval = action_build(emerge_config, spinner=spinner)
>   File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 354,
> in action_build
> success, mydepgraph, favorites = backtrack_depgraph(
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 10005, in backtrack_depgraph
> return _backtrack_depgraph(settings, trees, myopts, myparams,
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 10043, in _backtrack_depgraph
> success, favorites = mydepgraph.select_files(myfiles)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 4055, in select_files
> return self._select_files(args)
>   File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line
> 4189, in _select_files
> set_atoms = root_config.setconfig.getSetAtoms(s)
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 271, in getSetAtoms
> myatoms.update(self.getSetAtoms(s,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py",
> line 260, in getSetAtoms
> myatoms = myset.getAtoms()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> 58, in getAtoms
> self._load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line
> 53, in _load
> self.load()
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> 111, in load
> self._setAtoms(self.mapPathsToAtoms(self._files,
>   File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line
> 83, in mapPathsToAtoms
> for p in exclude_paths:
> TypeError: 'NoneType' object is not iterable
> 
> And i can't do anything else...
> 
> I see that python-3.8 has gone away
> # eselect python list
> Available Python interpreters, in order of preference:
>   [1]   python3.9
> 
> emerge --info file attached
> 
> I don't really know what to do.
> Any idea ?
> 
> Thanks a lot in advance,
> 
> Regards,
> 
> /--/
> /Jacques/

[gentoo-user] Python-3.9 and emerge problems

[Jacques Montier]

Hello all,

This morning :

#emerge --sync
#emerge --oneshot sys-apps/portage
#emerge -auvDN --with-bdeps=y --keep-going world

I get these errors :

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.9/emerge", line 51, in 
retval = emerge_main()
  File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in
emerge_main
return run_action(emerge_config)
  File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 3392, in
run_action
retval = action_build(emerge_config, spinner=spinner)
  File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 354, in
action_build
success, mydepgraph, favorites = backtrack_depgraph(
  File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 10005,
in backtrack_depgraph
return _backtrack_depgraph(settings, trees, myopts, myparams,
  File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 10043,
in _backtrack_depgraph
success, favorites = mydepgraph.select_files(myfiles)
  File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 4055,
in select_files
return self._select_files(args)
  File "/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 4189,
in _select_files
set_atoms = root_config.setconfig.getSetAtoms(s)
  File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
271, in getSetAtoms
myatoms.update(self.getSetAtoms(s,
  File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
271, in getSetAtoms
myatoms.update(self.getSetAtoms(s,
  File "/usr/lib/python3.9/site-packages/portage/_sets/__init__.py", line
260, in getSetAtoms
myatoms = myset.getAtoms()
  File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line 58,
in getAtoms
self._load()
  File "/usr/lib/python3.9/site-packages/portage/_sets/base.py", line 53,
in _load
self.load()
  File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line 111,
in load
self._setAtoms(self.mapPathsToAtoms(self._files,
  File "/usr/lib/python3.9/site-packages/portage/_sets/dbapi.py", line 83,
in mapPathsToAtoms
for p in exclude_paths:
TypeError: 'NoneType' object is not iterable

And i can't do anything else...

I see that python-3.8 has gone away
# eselect python list
Available Python interpreters, in order of preference:
  [1]   python3.9

emerge --info file attached

I don't really know what to do.
Any idea ?

Thanks a lot in advance,

Regards,

*--*
*Jacques*
Portage 3.0.20 (python 3.9.5-final-0, default/linux/amd64/17.1/desktop, 
gcc-10.3.0, glibc-2.33, 5.10.27-gentoo x86_64)
=
System uname: 
Linux-5.10.27-gentoo-x86_64-Intel-R-_Core-TM-_i7_CPU_950_@_3.07GHz-with-glibc2.33
KiB Mem:18428432 total,  12898428 free
KiB Swap:   18874364 total,  18874364 free
Timestamp of repository gentoo: Fri, 18 Jun 2021 09:00:01 +
Head commit of repository gentoo: e760f88d937bcf1db534f3294831835e9e45203e
sh bash 5.1_p8
ld GNU ld (Gentoo 2.35.2 p1) 2.35.2
app-shells/bash:  5.1_p8::gentoo
dev-java/java-config: 2.3.1::gentoo
dev-lang/perl:5.32.1::gentoo
dev-lang/python:  3.9.5_p2::gentoo
dev-lang/rust-bin:1.52.1::gentoo
dev-util/cmake:   3.18.5::gentoo
dev-util/pkgconfig:   0.29.2::gentoo
sys-apps/baselayout:  2.7::gentoo
sys-apps/openrc:  0.42.1-r1::gentoo
sys-apps/sandbox: 2.24::gentoo
sys-devel/autoconf:   2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:   1.11.6-r3::gentoo, 1.16.3-r1::gentoo
sys-devel/binutils:   2.35.2::gentoo
sys-devel/gcc:10.3.0::gentoo
sys-devel/gcc-config: 2.4::gentoo
sys-devel/libtool:2.4.6-r6::gentoo
sys-devel/make:   4.3::gentoo
sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers)
sys-libs/glibc:   2.33::gentoo
Repositories:

gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
sync-rsync-verify-jobs: 1
sync-rsync-verify-max-age: 24
sync-rsync-extra-opts: 
sync-rsync-verify-metamanifest: yes

overlay_local
location: /var/db/repos/gentoo/local
masters: gentoo
priority: 0

Installed sets: @kernels
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=corei7 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config 
/usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d 
/etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release 
/etc/php/apache2-php7.4/ext-active/ /etc/php/cgi-php7.4/ext-active/ 
/etc/php/cli-php7.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d 
/etc/terminfo"
CXXFLAGS="-march=corei7 -O2 -pipe"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB 
PERL5OPT PERLPREFIX PERL_CORE PERL_MB_O

Re: [gentoo-user] New install - root is mounted read-only

[Adam Carter]

On Fri, Jun 18, 2021 at 7:15 PM Wols Lists  wrote:

> I've started tackling my new build again, and when it boots root is
> read-only. Hopefully I've just missed something stupid, but how to I get
> it to transition read-write?
>

Weird - here's my fstab and dmesg entries if you want to compare;
# 
/dev/nvme0n1p1 /boot vfat noauto 1 2
/dev/nvme0n1p2 / ext4 noatime 0 1

dmesg | grep -i mount
[0.204991] Mount-cache hash table entries: 131072 (order: 8, 1048576
bytes, linear)
[0.205069] Mountpoint-cache hash table entries: 131072 (order: 8,
1048576 bytes, linear)
[2.390990] EXT4-fs (nvme0n1p2): mounted filesystem with ordered data
mode. Opts: (null). Quota mode: none.
[2.391735] VFS: Mounted root (ext4 filesystem) readonly on device 259:2.
[2.392955] devtmpfs: mounted
[2.660857] systemd[1]: Condition check resulted in Arbitrary Executable
File Formats File System Automount Point being skipped.
[2.691123] systemd[1]: Mounting Huge Pages File System...
[2.695813] systemd[1]: Mounting Kernel Debug File System...
[2.699613] systemd[1]: Mounting Kernel Trace File System...
[2.703758] systemd[1]: Mounting Temporary Directory (/tmp)...
[2.739488] systemd[1]: Mounted Huge Pages File System.
[2.741924] systemd[1]: Mounted Kernel Debug File System.
[2.744313] systemd[1]: Mounted Kernel Trace File System.
[2.746582] systemd[1]: Mounted Temporary Directory (/tmp).
[2.776402] systemd[1]: Mounting FUSE Control File System...
[2.782926] systemd[1]: Mounting Kernel Configuration File System...
[2.787158] systemd[1]: Starting Remount Root and Kernel File Systems...
[2.794343] systemd[1]: Mounted FUSE Control File System.
[2.796604] systemd[1]: Mounted Kernel Configuration File System.
[2.796823] EXT4-fs (nvme0n1p2): re-mounted. Opts: (null). Quota mode:
none.
[2.803599] systemd[1]: Finished Remount Root and Kernel File Systems.
[2.845843] systemd[1]: Set up automount mnt-backup.automount.
[2.848547] systemd[1]: Set up automount mnt-public.automount.

[gentoo-user] New install - root is mounted read-only

[Wols Lists]

I've started tackling my new build again, and when it boots root is
read-only. Hopefully I've just missed something stupid, but how to I get
it to transition read-write?

System is grub, systemd, and root is an lv ...

Do I need to do anything special with the initrd? Manually remounting
fixes it fine as far as I can tell.

Cheers,
Wol