On 04.03.22 00:46, Richard Purdie wrote:
On Thu, 2022-03-03 at 15:28 -0800, Tim Orling wrote:
On Thu, Mar 3, 2022 at 9:16 AM Richard Purdie
<richard.pur...@linuxfoundation.org> wrote:
On Thu, 2022-03-03 at 17:14 +0000, Ross Burton wrote:
On Thu, 3 Mar 2022 at 16:34, Richard Purdie
<richard.pur...@linuxfoundation.org> wrote:
This removes a further 1600 files from sstate handling and lets python
create the ones it wants at runtime which is likely much better overall
for performance.
Playing devil's advocate: doesn't having them in sstate mean they're
generated once and hardlinked, instead of needing to be generated for
every recipe which runs pythonnative?
Whilst I can't disagree that 1600 files being dropped from sstate is
good, we're just punting the recompile step to every recipe when it
runs python code.
I guess the question here is how long does the Python library take to
recompile.
At runtime it should only generate pyc for modules actually used?
Yes.
Another consideration is that there are many sysroots pulling in python3-
native
which don't run python and they're only there as there are python scripts
being
added which means python has to come too.
I suspect for that reason it could be a net win but it is a tough call.
I tend to be in favor of the reduced number of files moving around. I suppose
we could do some timing comparisons if we really care. It will be a trade-off
of network/io vs compute/io, correct?
Yes. I suspect the compile cost is tiny compared to the io/network cost of
chucking so many small files around when I suspect they're not used that often.
Cheers,
Richard
Now as this is in master and works reasonably well, could we extent the
removal of pyc for native recipes to *all* python packages.
I'm still seeing occasional staging issues on random native python
recipes as described in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13868.
And as the common understanding here seems to be that the additional
compile costs are negligible compared to the staging costs of additional
files, I would propose to add
do_install:append:class-native() {
find ${D} -name *.pyc -delete
}
to setuptools3.bbclass (as this looks like the one piece of code that
almost all python packages have in common)
Any thoughts?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162797):
https://lists.openembedded.org/g/openembedded-core/message/162797
Mute This Topic: https://lists.openembedded.org/mt/89528765/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
