On 12/10/2018 18:53, Cleber Rosa wrote: > TL;DR > ===== > > Allow acceptance tests to be run with `make check-acceptance`. > > Details > ======= > > This introduces a Python virtual environment that will be setup within > the QEMU build directory, that will contain the exact environment that > tests may require. > > There's one current caveat: it requires Python 3, as it's based on the > venv module. This was based on some discussions and perception about > standardizing on Python 3, but can easily be made to accommodate Python > 2 as well. > > Example of bootstrap and test execution on Travis-CI: > > https://travis-ci.org/qemu/qemu/jobs/439331028#L2508
If you activate Travis on your github account, you can test that in your namespace without having to open zombie pull requests there... A simple push to your repository will trigger a full Travis build. This is how I use it btw, canceling the jobs I'm not interested in, to quickly run the others. i.e. https://travis-ci.org/philmd/qemu/jobs/439573299#L5600 > > ... > VENV /home/travis/build/qemu/qemu/tests/venv > MKDIR /home/travis/build/qemu/qemu/tests/results > PIP /home/travis/build/qemu/qemu/tests/venv-requirements.txt > AVOCADO tests/acceptance > JOB ID : 920e4fcf55a1782f1ae77bee64b20ccdc2e1111d > JOB LOG : > /home/travis/build/qemu/qemu/tests/results/job-2018-10-09T21.42-920e4fc/job.log > (1/6) > /home/travis/build/qemu/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test: > PASS (3.57 s) > (2/6) > /home/travis/build/qemu/qemu/tests/acceptance/version.py:Version.test_qmp_human_info_version: > PASS (0.04 s) > (3/6) > /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS > (0.04 s) > (4/6) > /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: > PASS (0.04 s) > (5/6) > /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: > PASS (0.04 s) > (6/6) > /home/travis/build/qemu/qemu/tests/acceptance/vnc.py:Vnc.test_vnc_change_password: > PASS (0.04 s) > RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | > CANCEL 0 > JOB TIME : 3.90 s > ... > > Changes from v3: > ================ > > * Fixed typo in commit message (s/requiment/requirement/). (Eric) > > Changes from v2: > ================ > > * Make the $(TESTS_VENV_DIR) target depend on the > venv-requirements.txt file, and touch $(TESTS_VENV_DIR) after venv > runs. With this, updates on the file are reflected on the > venv. (Philippe) > > * Run pip with "python -m pip". It may have been installed reusing > the system wide packages, and then the script may not be available > on the venv. (Philippe) > > * Dropped Python version on Travis, and using the version supplied > by the distro (3.4). (Philippe) > > * Added "python3.4-venv" package requirement on Travis. (Philippe) > > * Added variable (AVOCADO_SHOW) with logging streams to be shown > while running the acceptance tests. By default it's set to none, > the equivalent of the quiet mode used on previous versions. > (Philippe) > > * On Travis, set the AVOCADO_SHOW variable to "app", so that the > individual test results can be easily seen. (Philippe) > > Ideas discussed, but not implemented: > > * Run pip with "$(PYTHON) -m pip -q install ..." because it points > to the system wide Python installation. (Philippe) > > * Drop the "--system-site-packages" flag. Waiting on another round > of tests to determine if they are really the cause of some package > installation problems. > > Changes from v1: > ================ > > * TESTS_VENV_REQ (the path of "venv-requirements.txt") now points to > the source path ($SRC_PATH instead of $BUILD_DIR) > > * Create the venv with "--system-site-packages", which allows the > reuse of packages (and no additional downloads) in case there's a > package installed system wide providing the same package and > version. > > * Run Avocado with "python -m avocado". It may have been installed > reusing the system wide packages, and then the script may not > be available on the venv. > > * Improved documentation describing the Python 3, venv and pip > requirements. > > * Updated avocado-framework requirement to latest released version > (65.0) > > * (New commit) Added support for running the acceptance tests on > Travis. > > Ideas discussed, but not implemented: > > * Install external packages such as python3-pip on Debian based > systems, deemed too invasive on developer's systems. > > * Allow the use of Python 2, and consequently the "virtualenv" > module. > > Cleber Rosa (3): > Bootstrap Python venv for tests > Acceptance tests: add make rule for running them > Travis support for the acceptance tests > > .travis.yml | 5 +++++ > docs/devel/testing.rst | 35 ++++++++++++++++++++++++++++++----- > tests/Makefile.include | 37 +++++++++++++++++++++++++++++++++++++ > tests/venv-requirements.txt | 4 ++++ > 4 files changed, 76 insertions(+), 5 deletions(-) > create mode 100644 tests/venv-requirements.txt >