On Wed, Oct 23, 2019 at 03:11:57PM -0600, Stephen Warren wrote: > On 10/23/19 12:17 PM, Tom Rini wrote: > > On Wed, Oct 23, 2019 at 12:04:30PM -0600, Stephen Warren wrote: > > > On 10/23/19 11:29 AM, Stephen Warren wrote: > > > > On 10/23/19 11:12 AM, Tom Rini wrote: > > > > > On Wed, Oct 23, 2019 at 11:03:38AM -0600, Stephen Warren wrote: > > > > > > On 10/23/19 10:58 AM, Tom Rini wrote: > > > > > > > On Wed, Oct 23, 2019 at 10:55:57AM -0600, Stephen Warren wrote: > > > > > > > > On 10/22/19 9:20 PM, Tom Rini wrote: > > > > > > > > > Now that we have moved to being based on pytest for > > > > > > > > > python3 we need to > > > > > > > > > make our test.py wrapper more robust in terms of only calling > > > > > > > > > python3 > > > > > > > > > rather than possibly finding and using python2. To > > > > > > > > > do this, change from > > > > > > > > > execvp()'ing pytest to invoking the package itself > > > > > > > > > via python. In the > > > > > > > > > event that pytest is unavailable we still get a user-friendly > > > > > > > > > error: > > > > > > > > > > > > > > > > I guess if this works, it's fine. But I think it'd work > > > > > > > > just as well to > > > > > > > > simply s/py.test/py.test-3/ in the existing code? > > > > > > > > > > > > > > We can't because pip only provides "py.test" / "pytest" binaries. > > > > > > > > > > > > Hmm. It sucks that virtualenv does things differently to Ubuntu > > > > > > packaging > > > > > > then. That means this series will probably break my test system, > > > > > > which > > > > > > doesn't use virtualenv. I guess I'll apply it and see. > > > > > > > > > > This should be what works everywhere tho. Instead of hoping that > > > > > "py.test" will be the right python version we tell python3 to use the > > > > > pytest module it has. So if you have system python3-pytest, that > > > > > works, > > > > > if you (python3) pip install pytest locally, that works and if you > > > > > virtualenv + python3 + pip install pytest, that works. > > > > > > > > On Ubuntu 16.04, with everything installed from Ubuntu packages rather > > > > than using virtualenv, I get this: > > > > > > > > > + ./src/u-boot/test/py/test.py --bd sandbox --build-dir > > > > > /var/lib/jenkins/workspace/u-boot-github_swarren_u-boot-tegra_dev-test-py/U_BOOT_BOARD/sandbox/build/u-boot/sandbox > > > > > -k 'not sleep' > > > > > Traceback (most recent call last): > > > > > File "./src/u-boot/test/py/test.py", line 20, in <module> > > > > > sys.exit(load_entry_point('pytest', 'console_scripts', > > > > > 'pytest')(args)) > > > > > File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", > > > > > line 542, in load_entry_point > > > > > return get_distribution(dist).load_entry_point(group, name) > > > > > File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", > > > > > line 2568, in load_entry_point > > > > > raise ImportError("Entry point %r not found" % ((group, name),)) > > > > > ImportError: Entry point ('console_scripts', 'pytest') not found > > > > > > > > > [swarren@swarren-lx1 u-boot]$ dpkg -l|grep py|grep test > > > > > ii python-pytest 2.8.7-4 > > > > > ii python3-pytest 2.8.7-4 > > > > > > > > I guess that's much older than the pytest in requirements.txt (5.2.1!!). > > > > While I think we can require a newish version of Python (3.5) since > > > > that's widely available on supported OSs, I don't think we want to force > > > > people to have a very new version of all the packages (which supported > > > > distros don't package yet), or change they way they run pytest (by > > > > requiring use of a virtualenv and requirements.txt) since that mechanism > > > > isn't supported on old branches of U-Boot, which will then require all > > > > kinds of special cases in scripts that invoke the U-Boot test system. > > > > > > Ah. The issue is the name of the entry point; the last parameter to > > > load_entry_point(). "py.test" is supported everywhere, but only newer > > > versions of pytesst have a "pytest" entry point. Changing that parameter > > > to > > > "py.test" makes the wrapper script work. Now every other test fails; I > > > will > > > see if there's an easy fix for that or not. > > > > We're going to have to make a cut-off point somewhere it looks like. > > pytest 2.8.7 (which we had pinned to) is incompatible with pytest 4.x > > and throws a lot of warnings I believe on pytest 3.x.. > > > > What I really want to avoid is the problem we have today where we've > > actively pinned ourselves to a now quite out of date version and have a > > lot of problems showing up. > > > > So we aren't going to have one environment that works for history and > > future but I am hoping / trying to plan such that if we have > > requirements.txt we can say from here until the next new best practices > > (which today would be using pipenv, but that starts out with pip install > > pipenv, sigh...) we can say to use requirements.txt and for history, > > python2 + pytest 2.8.7 and local setups could even dump that in to a > > requirements.txt file if that makes scripting easier. > > Indeed the pytest 4 changes were a lot of the problem, but there was plenty > more. It turned out to be too painful to fix it all. > > After a lot of pain, I've switched my system over to using a virtualenv when > there's a test/py/requirements.txt file in the source. Using a virtualenv > broke some of the HW remote control scripts, which were written for Python2 > but just said "#!/usr/bin/env python". Fixing that to request Python 2 > explicitly fixed that issue. Another pain was that the ftdi1 module (used by > the HW remote control scripts) isn't available from pip so can't be > installed in a virtualenv. Luckily explicitly using python2 for those > scripts makes this irrelevant, since system libraries are used, since the > python2 interpreter doesn't come from the virtualenv in that case. However, > if that didn't fix the issue, then I'd have needed extra options to set up > the virtualenv in order to rely on Ubuntu-provided package for the module. > I'm quoting them below just in case someone searching the archives finds > this later in the future! > > > +if [ -f ./src/u-boot/test/py/requirements.txt ]; then > > + rm -rf "${venv_dir}" > > + # --system-site-packages: Need ftdi1 module which is not available via > > pip, > > + # but is installed on the system > > + virtualenv --system-site-packages -p /usr/bin/python3 "${venv_dir}" > > + . "${venv_dir}/bin/activate" > > + # -U: upgrade: Ensure the package is installed in the venv, at the > > version > > + # requested in requirements.txt, even if already available on the > > host. > > + pip install -U -r ./src/u-boot/test/py/requirements.txt > > +fi > > So in summary, with the Python 3.5 fix I already sent, then this series, > Tested-by: Stephen Warren <swar...@nvidia.com>
I appreciate it, thanks! -- Tom
signature.asc
Description: PGP signature
_______________________________________________ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot