Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
Hi Andy, On Thu, 23 Dec 2021 at 02:07, Andy Shevchenko wrote: > > On Thu, Dec 23, 2021 at 7:08 AM Simon Glass wrote: > > On Tue, 14 Dec 2021 at 17:33, Simon Glass wrote: > > > > > > Importing libraries in Python caches the bytecode by default. > > > Since we run scripts in source tree it ignores the current directory > > > settings, which is $(srctree), and creates cache just in the middle > > > of the source tree. Move cache to the current directory. > > > > > > Signed-off-by: Andy Shevchenko > > > --- > > > v3: avoided crash (Simon), preserved tree hierarchy > > > tools/binman/main.py | 13 - > > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > > > Applied to u-boot-dm/next, thanks! > > > > I didn't notice this before, but this seems to create files like this: > > > > ./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc > > > > We don't really want to 'recache' the common Python files. Do you > > think we should revert this patch, or find another fix? > > I'm not sure I understand. efitools is not common, it's a separate > (non-standard) module and it's cached. The problem is that we don't want to create this file in the source directory. For me this file is in: /usr/lib/python3/dist-packages/elftools/ but it could be in ~/.local I suppose. In any case, the cache file should go there, shouldn't it? Regards, Simon
Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
On Thu, Dec 23, 2021 at 7:08 AM Simon Glass wrote: > On Tue, 14 Dec 2021 at 17:33, Simon Glass wrote: > > > > Importing libraries in Python caches the bytecode by default. > > Since we run scripts in source tree it ignores the current directory > > settings, which is $(srctree), and creates cache just in the middle > > of the source tree. Move cache to the current directory. > > > > Signed-off-by: Andy Shevchenko > > --- > > v3: avoided crash (Simon), preserved tree hierarchy > > tools/binman/main.py | 13 - > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > Applied to u-boot-dm/next, thanks! > > I didn't notice this before, but this seems to create files like this: > > ./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc > > We don't really want to 'recache' the common Python files. Do you > think we should revert this patch, or find another fix? I'm not sure I understand. efitools is not common, it's a separate (non-standard) module and it's cached. -- With Best Regards, Andy Shevchenko
Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
Hi Andy, On Tue, 14 Dec 2021 at 17:33, Simon Glass wrote: > > Importing libraries in Python caches the bytecode by default. > Since we run scripts in source tree it ignores the current directory > settings, which is $(srctree), and creates cache just in the middle > of the source tree. Move cache to the current directory. > > Signed-off-by: Andy Shevchenko > --- > v3: avoided crash (Simon), preserved tree hierarchy > tools/binman/main.py | 13 - > 1 file changed, 12 insertions(+), 1 deletion(-) > > Applied to u-boot-dm/next, thanks! I didn't notice this before, but this seems to create files like this: ./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc We don't really want to 'recache' the common Python files. Do you think we should revert this patch, or find another fix? Regards, Simon
Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
Importing libraries in Python caches the bytecode by default. Since we run scripts in source tree it ignores the current directory settings, which is $(srctree), and creates cache just in the middle of the source tree. Move cache to the current directory. Signed-off-by: Andy Shevchenko --- v3: avoided crash (Simon), preserved tree hierarchy tools/binman/main.py | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) Applied to u-boot-dm/next, thanks!
[PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
Importing libraries in Python caches the bytecode by default.
Since we run scripts in source tree it ignores the current directory
settings, which is $(srctree), and creates cache just in the middle
of the source tree. Move cache to the current directory.
Signed-off-by: Andy Shevchenko
---
v3: avoided crash (Simon), preserved tree hierarchy
tools/binman/main.py | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/tools/binman/main.py b/tools/binman/main.py
index 8c1e478d54ce..d19ded491af7 100755
--- a/tools/binman/main.py
+++ b/tools/binman/main.py
@@ -16,9 +16,20 @@ import sys
import traceback
import unittest
+# Get the absolute path to this file at run-time
+our_path = os.path.dirname(os.path.realpath(__file__))
+our1_path = os.path.dirname(our_path)
+our2_path = os.path.dirname(our1_path)
+
+#
+# Do not pollute source tree with cache files:
+# https://stackoverflow.com/a/60024195/2511795
+# https://bugs.python.org/issue33499
+#
+sys.pycache_prefix = os.path.relpath(our_path, os.environ.get('srctree',
our2_path))
+
# Bring in the patman and dtoc libraries (but don't override the first path
# in PYTHONPATH)
-our_path = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(2, os.path.join(our_path, '..'))
from patman import test_util
--
2.33.0
