[Avocado-devel] Avocado release 36.3 (LTS)
This is an announcement for the users of our Long Term Stability version of Avocado. This is a minor release that introduces bug fixes that are considered important to our users. Bugfixes * When checking for remote Avocado versions, ignore '\r'. This improves the stability of remote checks when error messages (usually from plugins) follow the version number given when running "avocado -v". * When sysinfo generates output that can not be decoded as UTF-8, the HTML report will not embed its contents. The user can still check the output file on the job results directory for the generated content. * The HTML report will also list the files that failed to be embedded into the report itself. * The local test runner loader will not attempt to discover tests when tests are to be run on a remote system and will *not* be copied to the remote system. Basically, if users gives the '--remote-no-copy' option, tests will not be attempted to be discovered locally. * The logging mechanisms in Avocado, which would replace the basics sys.stdout and sys.stderr could cause random failures resulting in crashes. Multiple users have reported different failures resulting in different crashes. In the end, this fix greatly increases the stability of Avocado. For a full list of changes, please refer to: https://github.com/avocado-framework/avocado/compare/36.2...36.3 LTS in a nutshell = The LTS releases have a special cycle that lasts for 18 months. Avocado usage in production environments should favor the use of this LTS release, instead of non-LTS releases. For more information, please refer to: https://www.redhat.com/archives/avocado-devel/2016-May/msg00025.html https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html Install Avocado === Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#installing-avocado Updated RPM packages are be available in the project repos for EPEL 6, EPEL 7, Fedora 23 (for the last time), Fedora 24 and now also for the newly released Fedora 25. Users subscribed to the LTS "channel", will get this 36.3 update, while users using the non-LTS repo, will probably be running 44.0 after an update. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ]
[Avocado-devel] Docs also available now on pythonhosted.org/avocado-framework
Hi folks, While working on "Check/Work around PIP upload failures"[1], I noticed that the "http://pythonhosted.org/avocado-framework"; URL is ours, and PyPI let us host documentation there. Instead of a 404, let's also upload the latest released docs[2] together with the released (code) tarball to PyPI. Thoughts? [1] - https://trello.com/c/w6dk6RDE/888-check-work-around-pip-upload-failures [2] - https://trello.com/c/hdh4w8WJ/890-upload-docs-to-pypi-keep-http-pythonhosted-org-avocado-framework-with-content -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Docs also available now on pythonhosted.org/avocado-framework
On 01/12/2017 12:28 PM, Lucas Meneghel Rodrigues wrote: > That's a great idea. Is the upload of the docs made through ssh? > I've done the first upload through the main project a web page[1]. There may be other ways to do the push. Ideally, it'd all flow through the usual setuptools commands (python setup.py register/upload/etc), but I'm not sure that's possible with our slightly tweaked "PKG-INFO" file. [1] https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=avocado-framework - Cleber. > On Thu, Jan 12, 2017 at 3:26 PM, Cleber Rosa wrote: >> Hi folks, >> >> While working on "Check/Work around PIP upload failures"[1], I noticed >> that the "http://pythonhosted.org/avocado-framework"; URL is ours, and >> PyPI let us host documentation there. >> >> Instead of a 404, let's also upload the latest released docs[2] together >> with the released (code) tarball to PyPI. >> >> Thoughts? >> >> [1] - >> https://trello.com/c/w6dk6RDE/888-check-work-around-pip-upload-failures >> [2] - >> https://trello.com/c/hdh4w8WJ/890-upload-docs-to-pypi-keep-http-pythonhosted-org-avocado-framework-with-content >> >> -- >> Cleber Rosa >> [ Sr Software Engineer - Virtualization Team - Red Hat ] >> [ Avocado Test Framework - avocado-framework.github.io ] >> > > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Docs also available now on pythonhosted.org/avocado-framework
On 01/12/2017 12:48 PM, Ademar Reis wrote: > On Thu, Jan 12, 2017 at 12:26:09PM -0200, Cleber Rosa wrote: >> Hi folks, >> >> While working on "Check/Work around PIP upload failures"[1], I noticed >> that the "http://pythonhosted.org/avocado-framework"; URL is ours, and >> PyPI let us host documentation there. >> >> Instead of a 404, let's also upload the latest released docs[2] together >> with the released (code) tarball to PyPI. >> >> Thoughts? > > Hi Cleber. > > Duplicating content on the web is not a good practice and will > hurt our search engine ranking. > > Instead, I recommend a temporary redirection (302 or 307). > Haven't thought of that. Indeed a 302 is better than a 404, which was my ultimate goal. The problem is that since we can only upload static pages, we're left with HTML meta tags and/or JavaScript to do the redirection... not as clean as a proper server side response. But this is what we have to work with, so this is what we ought to use. I'll experiment with that. Thanks for the suggestion! - Cleber. > Thanks. >- Ademar > >> >> [1] - >> https://trello.com/c/w6dk6RDE/888-check-work-around-pip-upload-failures >> [2] - >> https://trello.com/c/hdh4w8WJ/890-upload-docs-to-pypi-keep-http-pythonhosted-org-avocado-framework-with-content >> >> -- >> Cleber Rosa >> [ Sr Software Engineer - Virtualization Team - Red Hat ] >> [ Avocado Test Framework - avocado-framework.github.io ] >> > > > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Sprint #45 Review/Release (and #46 Planning) Meeting
Dear Avocado users and developers, I'd like to invite you all to a sprint release and planning meeting. It is going to take place at `date -d "Mon Jan 16 11:00:00 BRST 2017"` using the Hangouts on Air: https://www.youtube.com/watch?v=4LxdWAfnQB4 You can decide whether you want to join directly, or just watch the streaming and ask questions via IRC irc://irc.oftc.net/#avocado The meeting will be split in two parts (roughly 30 min each): * Part 1: Sprint Review * Review of the changes introduced during this sprint * Short demonstrations of some of the new features * Live sprint status: release readiness, last minute blockers, etc. * Part 2: Sprint Planning * New sprint planning boards will be created and its tasks prioritized. * Quick review of the expectations for the next sprint (high-level vision, goals, highlights) The meeting is going to be recorded and shared on Youtube afterwards. It is worth mentioning that most of the tasks during the meeting are going to take place on our Trello board: https://trello.com/b/WbqPNl2S/avocado See you there! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] (Pre) Release #45 Test Plan Results - FAIL on signed-off and spell checks
Results are mostly PASS, with the exception of the signed-off check (I'm to blame on latest merge commit) and spell checked (addressed by PR #1706). Test Plan: Release Test Plan Run by 'cleber' at 2017-01-16T07:19:52.297517 FAIL: 'Avocado source is sound': Signed-off check for the latest commit FAIL: 'Avocado source does not contain spelling errors': Addressed the spelling errors in PR #1706 PASS: 'Avocado RPM build': PASS: 'Avocado RPM lint': PASS: 'Avocado RPM install': PASS: 'Avocado Test Run on RPM based installation': PASS: 'Avocado Test Run on Virtual Machine': PASS: 'Avocado Test Run on Remote Machine': PASS: 'Avocado Remote Machine HTML report': PASS: 'Avocado Server Source Checkout and Unittests': PASS: 'Avocado Server Run': PASS: 'Avocado Server Functional Test': PASS: 'Avocado Virt and VT Source Checkout': PASS: 'Avocado Virt Bootstrap': PASS: 'Avocado Virt Boot Test Run and HTML report': PASS: 'Avocado Virt - Assignment of values from the cmdline': PASS: 'Avocado Virt - Migration test': PASS: 'Avocado VT': PASS: 'Avocado HTML report sysinfo': PASS: 'Avocado HTML report links': PASS: 'Paginator': avocado: ce0a2abb97756f700799a0f34050e4b194ecb6b2 avocado-server: 8cec494edcf160a8bc9f61e174ff7bcedb1e6800 avocado-virt: fb1e3b123e57e5f4f0527045de5a4bc653031e91 avocado-virt-tests: cc185f45cdadef0898f6a382ae81179cdfa1e5df avocado-vt: a4dc361c33640020c3d4dca1da658bc98a8b8e39 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 45.0: Anthropoid
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/45.0/release_notes/45_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 6, EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] What is a right way to install avocado?
Hi Andrei, I missed the "Workstation" variant for RHEL6. This is an easy fix, that should already be online. Thanks for reporting. - Cleber. - Original Message - > From: "Andrei Stepanov" > To: "avocado-devel" > Sent: Thursday, February 2, 2017 11:03:07 AM > Subject: Re: [Avocado-devel] What is a right way to install avocado? > > When I add https://repos-avocadoproject.rhcloud.com/static/avocado-el.repo > to RHEL6 I get: > > https://repos-avocadoproject.rhcloud.com/static/epel-6Workstation-noarch/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not > Found" > > > > On Wed, Feb 1, 2017 at 5:48 PM, Andrei Stepanov wrote: > > > Hello. > > > > We are currently experiencing some issues with avocado / avocado-vt. > > > > Our automation can be described in next steps: > > > > 0. Install RHEL 6/7. > > 1. Clone "master" branches for avocado/avocado-vt from github. > > 2. In avocado dir: > > > > make requirements > > python setup.py install > > > > 3. In avocado-vt dir: > > make link > > pip install sphinx > > pip install -r requirements.txt > > python setup.py install > > > > 4. Run tests. > > > > Above commands are run from root account. > > We cannot use this approach any more. > > It doesn't work with RHEL7.3. > > > > I have opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1417613 > > Than I had discussion with Tomas Orsava. > > > > The problem is, running pip as root in Fedora/EPEL is not supported and > > will break your system. > > > > https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe > > > > My question is: what is official way to install avocado/avocado-vt? > > > > Invoking pip commands from root account is a bad approach. > > > > Is there a safe way to install avocado & avocado-vt? > > >
Re: [Avocado-devel] What is a right way to install avocado?
- Original Message - > From: "Andrei Stepanov" > To: "avocado-devel" > Sent: Wednesday, February 1, 2017 11:48:11 AM > Subject: [Avocado-devel] What is a right way to install avocado? > > Hello. > > We are currently experiencing some issues with avocado / avocado-vt. > > Our automation can be described in next steps: > > 0. Install RHEL 6/7. > 1. Clone "master" branches for avocado/avocado-vt from github. Hi Andrei, Do you need specific features of Avocado not present in the LTS version? I would strongly recommend that for "production testing", you'd use a more stable version of Avocado. If you're willing to take a look at this suggested approach, let me if the fix for the Workstation version of EPEL6 works for you. > 2. In avocado dir: > > make requirements > python setup.py install > > 3. In avocado-vt dir: > make link > pip install sphinx > pip install -r requirements.txt > python setup.py install > For avocado-vt, an RPM package is also available (non-LTS, but designed to work with avocado LTS). Most dependencies would be solved by the package install alone. Then, dependencies for the test provider, say tp-qemu, could also be installed alongside it (but manually specified). > 4. Run tests. > > Above commands are run from root account. > We cannot use this approach any more. > It doesn't work with RHEL7.3. > > I have opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1417613 > Than I had discussion with Tomas Orsava. > > The problem is, running pip as root in Fedora/EPEL is not supported and > will break your system. > > https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe > > My question is: what is official way to install avocado/avocado-vt? > > Invoking pip commands from root account is a bad approach. > > Is there a safe way to install avocado & avocado-vt? >
Re: [Avocado-devel] avocado-vt: pull request 885, please merge it.
Hey Andrei, I'm CC'ing Xu and Su Qin and asking them to expedite the review of that PR. Xu and Su Qin, if you could spare some time to review and test PR #885 it would really help the Spice team. Thanks! - Cleber. On 02/02/2017 04:39 AM, Andrei Stepanov wrote: > Hi. > > Could you please run your tests with applied: > > https://github.com/avocado-framework/avocado-vt/pull/855 > > If it doesn't broke your tests then I would ask you to merge it. > > Thank you. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] What is a right way to install avocado?
On 02/03/2017 09:03 AM, Andrei Stepanov wrote: > Hello. > > Cleber Rosa: thank you for RHEL6 Workstation Variant. > You're welcome. Actually, thank you for reporting it. > 1. RPM. There is a hidden pitfall for all testers. EPEL is supposed to > be used on the latest RHEL/Centos. Yes. It is true. > Think about it as: EPEL should be installed on .Z stream. > It could be a problem for automation. For example. Some tester could be > asked to run avocado tests for package from RHEL 7.1. (Yes, QE teams > runs test against old RHEL.) Where current RHEL is 7.3. Than EPEL > supposed to be installed on RHEL 7.3. It could be a problem to install > EPEL on 7.1 (without updating packages to 7.3). Keep in mind that a > tester is asked to test 7.1 not 7.3 package. > So. My point is: in future minimize EPEL dependency. LTS avocado should > be easily installed on RHEL 7.0 7.1 7.2 7.3 7.3. > This (EPEL dependency) is essentially a trade off we had to decide on, with the other options at the time being bundling libraries (a real problem) or dropping features that could only be delivered with the help of external libraries. But, let me ensure you that we're working towards a much leaner (and eventually EPEL free) version of Avocado. As a matter of fact, this is the goal of the plugin based architecture we've invested on. This, for instance, is a WiP (close to being pushed upstream): ls -lh BUILD/RPM/ total 1.8M -rw-rw-r--. 1 cleber mock 404K Feb 1 07:35 avocado-45.0-0.fc25.noarch.rpm -rw-rw-r--. 1 cleber mock 704K Feb 1 07:32 avocado-45.0-0.fc25.src.rpm -rw-rw-r--. 1 cleber mock 228K Feb 1 07:35 avocado-examples-45.0-0.fc25.noarch.rpm -rw-rw-r--. 1 cleber mock97K Feb 1 07:35 avocado-plugins-output-html-45.0-0.fc25.noarch.rpm -rw-rw-r--. 1 cleber mock19K Feb 1 07:35 avocado-plugins-runner-docker-45.0-0.fc25.noarch.rpm -rw-rw-r--. 1 cleber mock27K Feb 1 07:35 avocado-plugins-runner-remote-45.0-0.fc25.noarch.rpm -rw-rw-r--. 1 cleber mock23K Feb 1 07:35 avocado-plugins-runner-vm-45.0-0.fc25.noarch.rpm It means that the "avocado" package will require as little as possible external packages (with the goal being no packages outside base RHEL), and the avocado-plugins-* packages with further requirements. > 2. Maybe it is better to figure out how to install Avocado in Python > Virtual Environment. This is more appropriate approach for me. As we can > flexible change git repo. It is necessary to investigate this > approach. https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe > <https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe> > Avocado should be usable from user&root accounts. > > Avocado itself should run just fine on a virtualenv. If there are problems, I'd love to hear about them. Also true with regards to running it as either a regular user or root. Avocado-VT, on the other hand, is to the best of my knowledge UNTESTED in a virtualenv. I recommend testing it and reporting specific problems. > 3. I think, our conversation should have some outcome. Some statement. > QE teams must clearly understand how to use Avocado. QE teams should > know where to report bugs related to Avocado installation. This should > be specified on main > page: https://github.com/avocado-framework/avocado/blob/master/README.rst > <https://github.com/avocado-framework/avocado/blob/master/README.rst> > . Something like: Officially supported OS is RHEL 6/x 7.x. Installation > bugs report at: https://github.com/avocado-framework/avocado/issues > <https://github.com/avocado-framework/avocado/issues> > I agree. And this is a key point where the various QE teams can help us (non-QE Avocado developers). We currently have the understanding that most deployment scenarios are Avocado LTS from packages + Avocado-VT from GIT. About the packages, we do *officially* support the packages we build and distribute. Those cover Fedora: http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#fedora And EL: http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#enterprise-linux Please keep in mind that: * For EL, as mentioned earlier, EPEL is (still) a requirement * We develop those packages on x86_64. This is what we can support at this point. > > 4. I would like to mention a RPM packaging shortcoming. > After installation avocado should work with: /var/lib/ > Currently it works with: > > avocado bootstrap command: > /usr/share/avocado/data/avocado-vt/test-providers.d/downloads/io-github-spiceqa-spice/spice/cfg/run.cfg > -- fetched from git > > avocado run command: > /usr/share/avocado/data/avocado-vt/backends/spice/cfg/run.cfg > > Please look at: > https://fedoraproject.org/wiki/Archive:PackagingDrafts/RPMMacros_sharedstatedir_optflags_and_admonitions > <
Re: [Avocado-devel] What is a right way to install avocado?
On 02/03/2017 09:39 AM, Lucas Meneghel Rodrigues wrote: > There is a tension between RPM being the one true way to install > software in Fedora, RHEL and derivatives and installing from pip, which > should be convenient and portable to any distro that has virtualenv. The > following use cases should be covered IMHO: > > 1) Full distro packages (avocado + avocado-vt coming from .rpms or .debs > or whatever else people would like to package avocado in) I believe Avocado-VT is a lot closer to their tests than Avocado itself (just think of the small amount of core test APIs Avocado makes available). I'd say it's much more common for tests to require changes to Avocado-VT than Avocado tests require changes to Avocado. That's why I believe Avocado-VT from GIT is quite common. Avocado, when used with Avocado-VT is much more a runtime of sorts. That's why, in my understanding, it makes sense get it from more stable sources. Either LTS packages or the LTS branch seem to be ideal. It should make no difference to Avocado-VT if Avocado comes from an RPM package or from source (pip, setup.py, etc), just as for Avocado itself, it should make no difference where the libraries it depends on came from. So yes, we're in sync when it comes to pip installs: it should work and it's considered an official installation procedure. The issue is that we've not being testing *Avocado-VT* on virtualenvs AFAICT. > 2) Full PIP install on a virtualenv (it should be mostly clean, except > for the external dependencies of avocado-vt) > Is anybody out there using such a deployment strategy for Avocado-VT? I'd love to hear about it. > I'm not sure how interesting it would be supporting 'mixed' setups. > Also, these days we have people experimenting with new ways to deploy > software on Linux, such as Flatpak. Maybe avocado could jump into the > gravy train as well. > For Flatpak, it would be great to have something in "contrib". Any volunteers? - Cleber. > On Fri, Feb 3, 2017 at 3:32 PM Cleber Rosa <mailto:cr...@redhat.com>> wrote: > > > On 02/03/2017 09:03 AM, Andrei Stepanov wrote: > > Hello. > > > > Cleber Rosa: thank you for RHEL6 Workstation Variant. > > > > You're welcome. Actually, thank you for reporting it. > > > 1. RPM. There is a hidden pitfall for all testers. EPEL is > supposed to > > be used on the latest RHEL/Centos. Yes. It is true. > > Think about it as: EPEL should be installed on .Z stream. > > It could be a problem for automation. For example. Some tester > could be > > asked to run avocado tests for package from RHEL 7.1. (Yes, QE teams > > runs test against old RHEL.) Where current RHEL is 7.3. Than EPEL > > supposed to be installed on RHEL 7.3. It could be a problem to install > > EPEL on 7.1 (without updating packages to 7.3). Keep in mind that a > > tester is asked to test 7.1 not 7.3 package. > > So. My point is: in future minimize EPEL dependency. LTS avocado > should > > be easily installed on RHEL 7.0 7.1 7.2 7.3 7.3. > > > > This (EPEL dependency) is essentially a trade off we had to decide on, > with the other options at the time being bundling libraries (a real > problem) or dropping features that could only be delivered with the help > of external libraries. > > But, let me ensure you that we're working towards a much leaner (and > eventually EPEL free) version of Avocado. As a matter of fact, this is > the goal of the plugin based architecture we've invested on. This, for > instance, is a WiP (close to being pushed upstream): > > ls -lh BUILD/RPM/ > total 1.8M > -rw-rw-r--. 1 cleber mock 404K Feb 1 07:35 > avocado-45.0-0.fc25.noarch.rpm > -rw-rw-r--. 1 cleber mock 704K Feb 1 07:32 > avocado-45.0-0.fc25.src.rpm > -rw-rw-r--. 1 cleber mock 228K Feb 1 07:35 > avocado-examples-45.0-0.fc25.noarch.rpm > -rw-rw-r--. 1 cleber mock97K Feb 1 07:35 > avocado-plugins-output-html-45.0-0.fc25.noarch.rpm > -rw-rw-r--. 1 cleber mock19K Feb 1 07:35 > avocado-plugins-runner-docker-45.0-0.fc25.noarch.rpm > -rw-rw-r--. 1 cleber mock27K Feb 1 07:35 > avocado-plugins-runner-remote-45.0-0.fc25.noarch.rpm > -rw-rw-r--. 1 cleber mock23K Feb 1 07:35 > avocado-plugins-runner-vm-45.0-0.fc25.noarch.rpm > > It means that the "avocado" package will require as little as possible > external packages (with the goal being no packages outside base RHEL), > and the avocado-plugins-* packages with further requirements.
[Avocado-devel] Heads-up: dropping Python 2.6 support
Hello to all Avocado users and developers, This is a heads-up about an important change that is coming to Avocado: we're dropping Python 2.6 support. Why? Initially, we planned Avocado to be supported on Python 2.7 and Python 3.x. Then we realized that a lot of our users still depended on Python 2.6 because of platforms such as EL6. Python 2.6 support came, and after a while our first LTS (36.0) version was released. This gives EL6 users a stable version they can rely on. Now it's time to look forward. By dropping official support for Python 2.6, we can focus our energy on other goals. Since we're talking Python versions, one of those if to support Python 3.x in the same code base. When? = Pretty soon, that is, as early as the published PR is accepted: https://github.com/avocado-framework/avocado/pull/1748 Users on EL6: = We recommend our users running Avocado on EL6 to stick to the LTS releases (currently version 36.3). This will ensure that bugfixes will be delivered to you. For more information on how to use the LTS versions please check the following link: http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux For more information of LTS releases, check the following link: http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux Alternatively, if you need features not present in the 36.0 series, you may still use a version as recent as 45.0, but *without* the level of support that LTS releases have. Please let us know if you have any issues or questions. Thanks! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Heads-up: dropping Python 2.6 support
On 02/06/2017 08:34 AM, Cleber Rosa wrote: > Hello to all Avocado users and developers, > > This is a heads-up about an important change that is coming to Avocado: > we're dropping Python 2.6 support. > > Why? > > > Initially, we planned Avocado to be supported on Python 2.7 and Python > 3.x. Then we realized that a lot of our users still depended on Python > 2.6 because of platforms such as EL6. > > Python 2.6 support came, and after a while our first LTS (36.0) version > was released. This gives EL6 users a stable version they can rely on. > > Now it's time to look forward. By dropping official support for Python > 2.6, we can focus our energy on other goals. Since we're talking Python > versions, one of those if to support Python 3.x in the same code base. > > When? > = > > Pretty soon, that is, as early as the published PR is accepted: > > https://github.com/avocado-framework/avocado/pull/1748 > > Users on EL6: > = > > We recommend our users running Avocado on EL6 to stick to the LTS > releases (currently version 36.3). This will ensure that bugfixes will > be delivered to you. For more information on how to use the LTS > versions please check the following link: > > http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux > > For more information of LTS releases, check the following link: > The proper link with info on LTS releases is: https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html Regards, - Cleber. > http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux > > Alternatively, if you need features not present in the 36.0 series, you > may still use a version as recent as 45.0, but *without* the level of > support that LTS releases have. > > Please let us know if you have any issues or questions. > > Thanks! > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Heads-up: dropping Python 2.6 support
On 02/07/2017 06:46 AM, Lucas Meneghel Rodrigues wrote: > I'm not sure putting a file with references to a relatively independent > plugin in the main avocado repo the correct thing to do. It's probably > better to put a file with that summary in the avocado-vt repository instead. > I think Andrei meant info about the 36lts branch (and 36.x release series), and not specifically about Avocado-VT. > On Tue, Feb 7, 2017 at 12:19 PM Andrei Stepanov <mailto:astep...@redhat.com>> wrote: > > Hi, > > How do you see an idea to create a file > in https://github.com/avocado-framework/avocado with information > from > https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html > ? > Mentioning the LTS branches (currently 36lts and their related releases) in indeed a good idea IMHO. Just a few lines, with a pointer to the proper (longer) explanation would suffice. - Cleber. > On Mon, Feb 6, 2017 at 2:41 PM, Cleber Rosa <mailto:cr...@redhat.com>> wrote: > > > On 02/06/2017 08:34 AM, Cleber Rosa wrote: > > Hello to all Avocado users and developers, > > > > This is a heads-up about an important change that is coming to > Avocado: > > we're dropping Python 2.6 support. > > > > Why? > > > > > > Initially, we planned Avocado to be supported on Python 2.7 > and Python > > 3.x. Then we realized that a lot of our users still depended > on Python > > 2.6 because of platforms such as EL6. > > > > Python 2.6 support came, and after a while our first LTS > (36.0) version > > was released. This gives EL6 users a stable version they can > rely on. > > > > Now it's time to look forward. By dropping official support > for Python > > 2.6, we can focus our energy on other goals. Since we're > talking Python > > versions, one of those if to support Python 3.x in the same > code base. > > > > When? > > = > > > > Pretty soon, that is, as early as the published PR is accepted: > > > > https://github.com/avocado-framework/avocado/pull/1748 > > > > Users on EL6: > > = > > > > We recommend our users running Avocado on EL6 to stick to the LTS > > releases (currently version 36.3). This will ensure that > bugfixes will > > be delivered to you. For more information on how to use the LTS > > versions please check the following link: > > > > > > http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux > > > > For more information of LTS releases, check the following link: > > > > The proper link with info on LTS releases is: > > https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html > > Regards, > - Cleber. > > > > > http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#enterprise-linux > > > > Alternatively, if you need features not present in the 36.0 > series, you > > may still use a version as recent as 45.0, but *without* the > level of > > support that LTS releases have. > > > > Please let us know if you have any issues or questions. > > > > Thanks! > > > > -- > Cleber Rosa > [ Sr Software Engineer - Virtualization Team - Red Hat ] > [ Avocado Test Framework - avocado-framework.github.io > <http://avocado-framework.github.io> ] > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Heads-up: dropping Python 2.6 support
File "avocado/core/job.py", line 34, in
> from . import runner
> File "avocado/core/runner.py", line 28, in
> from . import test
> File "avocado/core/test.py", line 31, in
> from . import multiplexer
> File "avocado/core/multiplexer.py", line 27, in
> from . import tree
> File "avocado/core/tree.py", line 53, in
> from . import output
> File "avocado/core/output.py", line 30, in
> import logutils
> ImportError: No module named logutils
>
>
> # pip install logutils
>
> Another error:
>
>
> # pip install 'avocado-framework==36.0'
> DEPRECATION: Python 2.6 is no longer supported by the Python core team,
> please upgrade your Python. A future version of pip will drop support
> for Python 2.6
> Collecting avocado-framework==36.0
> Using cached avocado-framework-36.0.tar.gz
> Complete output from command python setup.py egg_info:
> Traceback (most recent call last):
> File "", line 1, in
> File "/tmp/pip-build-e6YcVH/avocado-framework/setup.py", line 22,
> in
> from avocado import VERSION
> File "avocado/__init__.py", line 19, in
> from avocado.core.job import main
> File "avocado/core/job.py", line 34, in
> from . import runner
> File "avocado/core/runner.py", line 28, in
> from . import test
> File "avocado/core/test.py", line 44, in
> import unittest2 as unittest
> ImportError: No module named unittest2
>
>
> # pip install unittest2
>
> Another error:
>
> # pip install 'avocado-framework==36.0'
> DEPRECATION: Python 2.6 is no longer supported by the Python core team,
> please upgrade your Python. A future version of pip will drop support
> for Python 2.6
> Collecting avocado-framework==36.0
> Using cached avocado-framework-36.0.tar.gz
> Running setup.py
> (path:/tmp/pip-build-J7PtEy/avocado-framework/setup.py) egg_info for
> package avocado-framework produced metadata for project name avocado.
> Fix your #egg=avocado-framework fragments.
> Building wheels for collected packages: avocado, avocado
> Running setup.py bdist_wheel for avocado ... done
> Stored in directory:
> /root/.cache/pip/wheels/6a/20/35/aa577931e4582057dbee3fd3250636ef75e7d5e503e98e5249
> Running setup.py bdist_wheel for avocado ... error
> Complete output from command /root/avocado/bin/python -u -c "import
> setuptools,
> tokenize;__file__='/tmp/pip-build-J7PtEy/avocado/setup.py';f=getattr(tokenize,
> 'open', open)(__file__);code=f.read().replace('\r\n',
> '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d
> /tmp/tmpvBUgBOpip-wheel- --python-tag cp26:
> Traceback (most recent call last):
> File "", line 1, in
> IOError: [Errno 2] No such file or directory:
> '/tmp/pip-build-J7PtEy/avocado/setup.py'
>
>
> Failed building wheel for avocado
> Running setup.py clean for avocado
> Complete output from command /root/avocado/bin/python -u -c "import
> setuptools,
> tokenize;__file__='/tmp/pip-build-J7PtEy/avocado/setup.py';f=getattr(tokenize,
> 'open', open)(__file__);code=f.read().replace('\r\n',
> '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
> Traceback (most recent call last):
> File "", line 1, in
> IOError: [Errno 2] No such file or directory:
> '/tmp/pip-build-J7PtEy/avocado/setup.py'
>
>
> Failed cleaning build dir for avocado
> Successfully built avocado
> Failed to build avocado
> Installing collected packages: avocado
> Successfully installed avocado-36.0lts
>
>
Andrei,
Running a command such as:
pip install -r
https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt
Would have probably saved you a lot of this hassle. This is documented
here:
http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools
>
> Now I want to use git repo
> : https://github.com/avocado-framework/avocado-vt/
>
> But, unsuccessful. H
>
Have you tried:
$ pip install -e
"git+git://github.com/avocado-framework/avocado-vt#egg=avocado-plugins-vt"
This would be the right syntax, but I can not guarantee that avocado-vt
installs fine from pip.
Regards,
- Cleber.
>
>
> On Tue, Feb 7, 2017 at 12:49 PM, Cleber Rosa <
Re: [Avocado-devel] Heads-up: dropping Python 2.6 support
On 02/07/2017 08:34 AM, Andrei Stepanov wrote: > > > On Tue, Feb 7, 2017 at 2:20 PM, Cleber Rosa <mailto:cr...@redhat.com>> wrote: > > > > > Andrei, > > Running a command such as: > > pip install -r > > https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt > > <https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt> > > Would have probably saved you a lot of this hassle. This is documented > here: > > > http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools > > <http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools> > > > > > Can I ask why we do not specify required packages for pip? > http://python-packaging.readthedocs.io/en/latest/dependencies.html > We do specify the most basic requirements (stevedore) on newer versions, it did not make into 36lts though. We can still improve it and release a new 36.x version. But, even with `install_requires`, some older versions of setuptools do not allow versions to be specified there, which would cause breakage. Then, there's the question of too many deps required by some plugins, such as remote (requires fabric), vm (requires libvirt). We decided to keep the list short and plugins just won't work without the extra deps. You may have noticed that we're working on splitting those plugins that bring extra deps out of the "core" of Avocado: https://github.com/avocado-framework/avocado/pull/1751 > It would be fine if next command install all dependency automatically: > pip install 'avocado-framework==36.0' > > It *would* be awesome. But as you might imagine, it is not *that* simple. Putting a `install_requires` and releasing a new 36.x (36.4 now) would be the first big improvement IMHO. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Heads-up: dropping Python 2.6 support
On 02/07/2017 09:53 AM, Andrei Stepanov wrote: > It seems that I can install: Avocado LTS using virtualenv + avocado-vt > from github as a plugin. > > avocado shows tp-qemu tests. > > My steps are: > > 1. Enable RHEL optional repo. > 2. yum install libvirt libvirt-devel xz-devel libffi-devel python-devel > openssl-devel openssl-devel "@Development tools" > 3. python get-pip.py --user > 4. ./.local/bin/pip install --user virtualenv > 5. ./.local/bin/virtualenv a1 > 6. . ./a1/bin/activate > 7. pip install -r > https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt > 8. pip install 'avocado-framework==36.0' > 9. pip install aexpect netaddr autotest > 10. pip install -e > "git+git://github.com/avocado-framework/avocado-vt#egg=avocado-plugins-vt > <http://github.com/avocado-framework/avocado-vt#egg=avocado-plugins-vt>" > 11. yum install yum install p7zip p7zip-plugins # From EPEL (can we > get away from p7zip ?) > 12. yum install qemu-kvm qemu-kvm-tools > 13. avocado vt-bootstrap --vt-no-downloads > 14. avocado list > > How do you think, is this right approach to use avocado + avocado-vt on > RHEL 6.x + RHEL 7.x? > > If it works (as you verified), then it's definitely (one) correct deployment scenario. If you find minor changes that can greatly improve deployment (such as changes to requirements.txt), we can consider them "installation bugs" and include them in future LTS releases. - Cleber. > On Tue, Feb 7, 2017 at 2:41 PM, Cleber Rosa <mailto:cr...@redhat.com>> wrote: > > > > On 02/07/2017 08:34 AM, Andrei Stepanov wrote: > > > > > > On Tue, Feb 7, 2017 at 2:20 PM, Cleber Rosa <mailto:cr...@redhat.com> > > <mailto:cr...@redhat.com <mailto:cr...@redhat.com>>> wrote: > > > > > > > > > > Andrei, > > > > Running a command such as: > > > > pip install -r > > > https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt > > <https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt> > > > <https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt > > <https://raw.githubusercontent.com/avocado-framework/avocado/36lts/requirements.txt>> > > > > Would have probably saved you a lot of this hassle. This is > documented > > here: > > > > > http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools > > <http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools> > > > <http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools > > <http://avocado-framework.readthedocs.io/en/45.0/GetStartedGuide.html#installing-from-standard-python-tools>> > > > > > > > > > > Can I ask why we do not specify required packages for pip? > > http://python-packaging.readthedocs.io/en/latest/dependencies.html > <http://python-packaging.readthedocs.io/en/latest/dependencies.html> > > > > We do specify the most basic requirements (stevedore) on newer versions, > it did not make into 36lts though. We can still improve it and release > a new 36.x version. > > But, even with `install_requires`, some older versions of setuptools do > not allow versions to be specified there, which would cause breakage. > > Then, there's the question of too many deps required by some plugins, > such as remote (requires fabric), vm (requires libvirt). We decided to > keep the list short and plugins just won't work without the extra deps. > > You may have noticed that we're working on splitting those plugins that > bring extra deps out of the "core" of Avocado: > > https://github.com/avocado-framework/avocado/pull/1751 > <https://github.com/avocado-framework/avocado/pull/1751> > > > It would be fine if next command install all dependency automatically: > > pip install 'avocado-framework==36.0' > > > > > > It *would* be awesome. But as you might imagine, it is not *that* > simple. Putting a `install_requires` and releasing a new 36.x (36.4 > now) would be the first big improvement IMHO. > > -- > Cleber Rosa > [ Sr Software Engineer - Virtualization Team - Red Hat ] > [ Avocado Test Framework - avocado-framework.github.io > <http://avocado-framework.github.io> ] > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] pre_command / post_command clarification
On 02/10/2017 05:01 AM, Andrei Stepanov wrote: > Hi. > > We need a reliable mechanism to notify Beaker server about start+result > of a avocado-vt test. As you know, avocado-vt test is one of the many > tests produced from cartesian config. We need to notify Beaker about > start+results of _each_ test. > > Currently we use: pre_command / post_command > > I discovered yesterday, that this commands are not called for FAILED > tests. Especially those have a error in syntax. > > In IRC Lukáš Doktor suggested to: > > 19:22astepano: Hello Andrei, the `post_command` is part of the > `env_process` during the postprocess which means it's one of the cleanup > steps the problem is when the postprocess fails before getting to > `post_command` it will not be executed. It all depends on many factors. > Anyway I don't know what all information do you need to produce your > results but I'd strongly recommend writing either `JobPostTests` plugin, > or if you need per-test granularity t > 19:23Also writing such plugin is really simple and there are > examples in our sources... > > So, I want to bring our conversation to this mail listing. > > I am not sure that such plugin will do job for avocado-vt tests. > Avacodo-vt tests generated from cartesian configs. > > Could you please suggest me the right approach to coupe with this issue? > > I need mechanism to call external program before & after _each_ > avocado-vt test. The program's environment should have variables: > TESTNAME / TESTRESULT. > Andrei, As a *very* brief answer, I'd say this looks like something that can be implemented as a `ResultEvents` plugin: * http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats * http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents You would write methods such as "start_test" and "end_test" to send the needed info to Beaker. This would be a feature generic to all Avocado supported tests (including Avocado-VT tests). - Cleber. > > Thank you! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Pre-release test plan results - almost PASS
On 02/13/2017 03:34 AM, Lukáš Doktor wrote: > Test Plan: Release Test Plan > Run by 'ldoktor' at 2017-02-13T08:07:41.864590 > > PASS: 'Avocado source is sound': > PASS: 'Avocado source does not contain spelling errors': > PASS: 'Avocado RPM build': > PASS: 'Avocado RPM lint': How about checking all packages? There are > some warning/errors in them... > PASS: 'Avocado RPM install': Using only avocado and later added > avocado-examples > PASS: 'Avocado Test Run on RPM based installation': avocado-examples is > necessary > PASS: 'Avocado Test Run on Virtual Machine': > PASS: 'Avocado Test Run on Remote Machine': > PASS: 'Avocado Remote Machine HTML report': > PASS: 'Avocado Server Source Checkout and Unittests': > PASS: 'Avocado Server Run': > PASS: 'Avocado Server Functional Test': > FAIL: 'Avocado Virt and VT Source Checkout': > https://github.com/avocado-framework/avocado/pull/1781 ^ Merged. > PASS: 'Avocado Virt Bootstrap': > FAIL: 'Avocado Virt Boot Test Run and HTML report': > https://github.com/avocado-framework/avocado-virt/pull/96 ^ Merged. > PASS: 'Avocado Virt - Assignment of values from the cmdline': > PASS: 'Avocado Virt - Migration test': > PASS: 'Avocado VT': The tests count is now 3382 and not "just" 1906 > PASS: 'Avocado HTML report sysinfo': > PASS: 'Avocado HTML report links': > PASS: 'Paginator': > > aexpect: aef81e79b597fb3633da4fd5c2a51a863adafbd2 > avocado: 32273ee4ed2aee888690103cbf52a7019138b397 > avocado-server: 9eab1c6822b090e71063723b8ef56de7805308f8 > avocado-virt: fb1e3b123e57e5f4f0527045de5a4bc653031e91 > avocado-virt-tests: 23e9f6ace369b09b6e53007e5dc759c18576914a > avocado-vt: e1c20088823d8480a8eff6f9c16fb77e1dd59301 > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Unexpected conversion of yaml content to ListOfNodeObjects
On 02/13/2017 03:33 AM, Vincent Matossian wrote:
>
> I'm seeing an odd behavior on v0.45, when trying to load a yaml file in
> an instrumented test the yaml content gets converted to a list of
> MuxTreeNodes rather than a dictionary.
>
> here's a simple example
>
> mytest.py
>
> #!/bin/env python
> import yaml
> from avocado import Test
>
> class MyTest(Test):
>
> def test(self):
> with open("some.yaml") as f:
> d = yaml.load(f)
> print(type(d))
> print(d)
>
>
> some.yaml:
>
> key1:
> subkey: subval
> key2:
> subkey: subval
>
> An empty.yaml file
>
> Run: avocado run --show-job-log mytest.py --mux-yaml empty.yaml
>
> Relevant output:
>
> START 1-d.py:MyTest.test
>
> [MuxTreeNode(name='key1'), MuxTreeNode(name='key2')]
> PASS 1-d.py:MyTest.test
>
> If I drop the mux-yaml option then the output looks like what I'd expect
>
> START 1-d.py:MyTest.test
>
> {'key2': {'subkey': 'subval'}, 'key1': {'subkey': 'subval'}}
> PASS 1-d.py:MyTest.test
>
>
On my machine:
$ avocado --show test run mytest.py | grep -e type -e key1
{'key2': {'subkey': 'subval'}, 'key1': {'subkey': 'subval'}}
$ avocado --show test run mytest.py --mux-yaml empty.yaml | grep -e type
-e key1
{'key2': {'subkey': 'subval'}, 'key1': {'subkey': 'subval'}}
That is, I can not reproduce it. I'm running avocado from latest master
(337b333e1b58f18f876c993121454f2f6cb599db). Can you share your own version?
> --mux-yaml seems to intercept all yaml conversions, this is not
> desirable in my case as I lose the ability to use yaml in my test, my
> apologies for not doing much research on whether this is a known, please
> let me know if I missed the obvious or how I can work around it
>
If this turns out to be reproducible, it may point at some serious
problems with Avocado: the runner influencing the "yaml" module loaded
in the test.
I really want to check if this is a current bug.
> Thanks
>
Thank you for reporting it.
> Vincent
>
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
signature.asc
Description: OpenPGP digital signature
[Avocado-devel] Avocado release 46.0: Burning Bush
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/46.0/release_notes/46_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/46.0/GetStartedGuide.html#installing-avocado Updated RPM packages will be available for EPEL 7, Fedora 24 and 25. As this release saw the removal of Python 2.6, EPEL 6 is no longer supported. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Subprocess termination
> mailto:apa...@redhat.com>> wrote: > >> >> > >> >> On Thu, Feb 16, 2017 at 4:38 PM, Radek Duda > mailto:rd...@redhat.com>> wrote: > >> >> > Dear avocado users and developers, > >> >> > I have made a testcase in which is executed nc process > in run > >> >> > function : > >> >> > (simplified version): > >> >> > > >> >> > from avocado.utils import process > >> >> > > >> >> > def run(vt_test, test_params, env): > >> >> > cmd = "nc -l %s" % test_params['some_port'] > >> >> > nc_process = process.SubProcess(cmd) > >> >> > return > >> >> > > >> >> > after testcase is executed, nc does not terminate and is > still > >> >> > present. > >> >> > To > >> >> > avoid this I have to kill the process e.g. by > >> >> > process.safe_kill(nc_process_pid, signal.SIGKILL) > >> >> > >> >> Are you running the process with `nc_process.run()`? It's > expected to > >> >> wait for the process to finish. > >> >> > >> >> > >> >> > >> >> > > https://github.com/avocado-framework/avocado/blob/master/avocado/utils/process.py#L596-L643 > > <https://github.com/avocado-framework/avocado/blob/master/avocado/utils/process.py#L596-L643> > >> >> > >> >> > >> >> > > >> >> > It is pretty awkward to close the process manually > particularly in > >> >> > case > >> >> > of > >> >> > complex testcase code > >> >> > Shouldn't be the subprocess killed automatically after > test exits? > >> >> > After all its called SubProcess > >> >> > > >> >> > > >> >> > regards, > >> >> > > >> >> > Radek Duda > >> >> > >> > > > > > > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Subprocess termination
On 02/20/2017 07:13 AM, Andrei Stepanov wrote: > Hi > > Cleber, I think your example is not completely correct. You use "jobs". > Bash/csh/zsh jobs is another topic. Your example is about bash jobs. > Let's take a look a level up: > > 1. open xterm/gnome-terminal. > 2. Run + put it in background: sleep 600 & > 3. Close the terminal. > 4. Check for processes. sleep is not disconnected. > > Right, the example was an analogy (not a technical deep dive of the implementation). It was intended to be didactic, and it looks like that goal was achieved. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Subprocess termination
On 02/20/2017 08:46 AM, Andrei Stepanov wrote: > It depends. > > There could be a scenario where necessary to run ~ 200 tests at once. > If first bad-by-design test forgets/failed to cleanup, then some > sequential tests also will fail. > It is a question about usability. > Andrei, I've personally thought about this some time ago. Right now, Avocado tries to "isolate" tests from each other by using separate processes. This is a rather weak isolation, but still better than what many other test runners offer. I believe that having stronger isolation at a test level would indeed be really useful. Running each test on a separate and disposable "execution environment" (container, vm with snapshot, physical machine that gets reloaded) would be really powerful. Still, we don't have that feature at this time... > > Okay, Radek, it seems that we should carefully write our tests, and kill > children processes no matter what of error we have in our tests., at > least for now. > ... and it doesn't exclude the careful writing of tests as a good practice. - Cleber. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Deprecation notice: Avocado-VT jeos-21-64 image
Dear Avocado-VT Users, This is a deprecation notice for the "jeos-21-64.qcow2" image. The "jeos-23-64.qcow2", based on Fedora 23, has been available for quite some time now, and instead of carrying such an old version on our pretty limited storage space, we plan to: * Change the compression format from 7z to xz * Introduce another image based on Fedora 25 If you have any questions, please let us know. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Promote Andrei to vt-maintainers group?
On 02/23/2017 05:50 AM, Lukáš Doktor wrote: > Hello guys, > > I see an increasing tension between tp-spice guys and the rest of > avocado-vt guys because some of their patches do not get enough > priority. Currently there is no maintainer from tp-spice group so to > improve the situation I'd like to promote Andrei Stepanov to > vt-maintainers group giving him write access. He is pretty active for a > long time and I believe he'll take this role responsibly. > > Please let me know about your opinions, > Lukáš > +1 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Make some avocado.utils.process APIs compatible with macOS
On 03/04/2017 07:36 AM, Lucas Meneghel Rodrigues wrote: > Let's discuss this particular topic a little bit. > > So I dedicated some time to make avocado self tests to run on macOS > (see https://github.com/avocado-framework/avocado/pull/1834). This would > be a first step to make macOS (new name of what was known as Mac OS X) a > 2nd tier supported platform. This would interesting from a project > health perspective since attracting macOS developers would expand user > base and mind share. > +1 on this. As mentioned during the meeting, even if we'd not attract other developers and users (which is certainly not the case), it'd probably be beneficial to the overall architecture and/or implementation of Avocado. > Now, some of the avocado.utils.process APIs (example, get_children_pids) > rely on executing 'ps' as a subprocess. We implemented those functions > contradicting our own rules of avoiding relying on subprocesses, but it > was the pragmatic choice at the moment. > Right. I'm looking at the exact places we do this, and quickly evaluating how hard it would be to get the same info from the more direct (and platform dependent) source. For instance, on `avocado.utils.process.kill_process_tree()` we have the execution of `"ps --ppid=%d -o pid=` which could be replaced by a `get_children_processes()` that would adapt to the supported platforms. > macOS does have coreutils, but they all come from BSD, not GNU. The > macOS version of 'ps' does not support the command line arguments that > make the APIs to return the correct values. It stands to reason that, if > we want to make avocado more portable among platforms, we should look > for, well, more portable implementations. > Agreed. > For that particular case, the excellent 3rd party 'psutil' library gives > us all the facilities implemented in a portable way so that it would > just work under macOS (and possibly other systems). Summarizing the choices: > The goal of making Avocado run on other platforms (such as macOS) may conflict with making Avocado run *easily* on many platforms, and this is a very good example. We have been working hard to minimize dependencies for the *basic* Avocado test runner, and since `avocado.utils.process` is used on it, it would be step in the opposite direction. > 1) Use 'psutil' but then introduce an EPEL dependency on RHEL platforms. > I know some of the groups using avocado have been pushing for a core > program that does not rely on EPEL, so for those guys, that solution is > a no-go. > Yep, a bad idea for a lot of people, as mentioned before. > 2) 'borrow' the psutil implementations and put them in avocado, which is > frowned upon by Fedora and RHEL and pretty much every distro packager > ever. I don't like that idea. > If we just need 2 or 3 portable functions, we can either borrow the idea (or if the licensing allows) the code itself. > 3) Remove the offending APIs. That would be a valid solution, although > capping existing functionality is not something that I fancy. We are > still with a liability of having avocado libs relying on parsing > external subprocess outputs, which is bad for a number of reasons and it > should be addressed at some point. > We can rewrite how Avacado currently does that on Linux, by using, say, the /proc filesystem. > 4) Simply say the APIs are not supported on macOS. It's also valid but > then it would be interesting to start delimiting linux specific APIs to > a namespace (think avocado.utils.linux, avocado.utils.darwin, > avocado.utils.nt) to make things clear from the get go. > -1 > 5) Forget about the whole 'support non-linux' idea. > -1 > Let me know your thoughts on the subject. > Let's try to identify what we *really* need to support here, and decide if they deserve a borrowed idea and/or implementation from psutil. I'd expect them to be fairly simple that we can write and maintain in Avocado itself. > Cheers, > > Lucas > > Thanks for you work on this! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 47.0: The Lost Wife
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/47.0/release_notes/47_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/47.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] python-aexpect (drop aexpect requirement and package from repo)
Hi Merlin, I'm resuming work on the removal of deltas between upstream and downstream packages, and one of the most urgent tasks I noticed isto get rid of our own aexpect package. I see that python-aexpect is built on all the following distros (at least): * EL7: https://koji.fedoraproject.org/koji/buildinfo?buildID=863392 * F24: https://koji.fedoraproject.org/koji/buildinfo?buildID=863388 * F25: https://koji.fedoraproject.org/koji/buildinfo?buildID=860495 Which is enough for what we need in Avocado "upstream" packages. But, the python-aexpect package has been pushed only to F25 so far. If we publish it to the other distros, I can eliminate the "aexpect" requirement from the upstream SPEC file. Is that feasible? Thanks! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] python-aexpect (drop aexpect requirement and package from repo)
On 03/08/2017 12:02 PM, Cleber Rosa wrote: > Hi Merlin, > > I'm resuming work on the removal of deltas between upstream and > downstream packages, and one of the most urgent tasks I noticed isto get > rid of our own aexpect package. > > I see that python-aexpect is built on all the following distros (at least): > > * EL7: https://koji.fedoraproject.org/koji/buildinfo?buildID=863392 > * F24: https://koji.fedoraproject.org/koji/buildinfo?buildID=863388 > * F25: https://koji.fedoraproject.org/koji/buildinfo?buildID=860495 > > Which is enough for what we need in Avocado "upstream" packages. But, > the python-aexpect package has been pushed only to F25 so far. > > If we publish it to the other distros, I can eliminate the "aexpect" > requirement from the upstream SPEC file. > > Is that feasible? > > Thanks! > BTW, the following PR was merged on aexpect: https://github.com/autotest/aexpect/pull/17 Which renames the package to python-aexpect. I have pushed an updated version of it to the upstream YUM/DNF repo, and have dropped the "aexpect" packages from it. This will make things easier on the avocado SPEC file. The next step is to drop the custom aexpect packages from the upstream repo. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] python-aexpect (drop aexpect requirement and package from repo)
On 03/08/2017 04:35 PM, Merlin Mathesius wrote: > > On 03/08/2017 01:39 PM, Cleber Rosa wrote: >> On 03/08/2017 12:02 PM, Cleber Rosa wrote: >>> Hi Merlin, >>> >>> I'm resuming work on the removal of deltas between upstream and >>> downstream packages, and one of the most urgent tasks I noticed isto get >>> rid of our own aexpect package. >>> >>> I see that python-aexpect is built on all the following distros (at least): >>> >>> * EL7: https://koji.fedoraproject.org/koji/buildinfo?buildID=863392 >>> * F24: https://koji.fedoraproject.org/koji/buildinfo?buildID=863388 >>> * F25: https://koji.fedoraproject.org/koji/buildinfo?buildID=860495 >>> >>> Which is enough for what we need in Avocado "upstream" packages. But, >>> the python-aexpect package has been pushed only to F25 so far. >>> >>> If we publish it to the other distros, I can eliminate the "aexpect" >>> requirement from the upstream SPEC file. >>> >>> Is that feasible? > Sure thing. I just submitted requests to publish the python-aexpect > package for the following distros: > > * F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-94bb07357b > * EL7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-5afaa778a4 > > Once the packages hit the testing repos, feel free to check them out and > give karma. Thanks a lot! I will give, buy and ask for karma donations! :) >>> Thanks! >>> >> BTW, the following PR was merged on aexpect: >> >> https://github.com/autotest/aexpect/pull/17 >> >> Which renames the package to python-aexpect. I have pushed an updated >> version of it to the upstream YUM/DNF repo, and have dropped the >> "aexpect" packages from it. >> >> This will make things easier on the avocado SPEC file. >> >> The next step is to drop the custom aexpect packages from the upstream repo. >> > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Fedora 25 and JeOS 25 Image sizes
Hi Avocado and Avocado-VT users, I promised some data about the image sizes for Fedora 25 and JeOS 25, with and without Python 2 and Avocado installed. Here they are: Fedora 25 (- Python 2, - Avocado): 2189426688 Fedora 25 (+ Python 2, - Avocado): 2232418304 (+ ~41 MiB) Fedora 25 (+ Python 2, + Avocado 46.0): 2240741376 (+ ~7 MiB) Fedora 25 (+ Python 2, + Avocado 47.0): 2268659712 (+ ~26 MiB)[1] JeOS 25 (-Python 2, - Avocado 47.0): 940703744 200683476 (xz) JeOS 25 (+Python 2, - Avocado 47.0): 985333760 (+ ~42 MiB) 207663204 (xz, + ~6 MiB) JeOS 25 (+Python 2, + Avocado 47.0): 1215102976 (+ ~219 MiB) 262776456 (xz, + ~52 MiB) A short summary of these experiments is that Python 2 and Avocado add very little to the Fedora 25 installs (as proposed on PR #926)[2]. For JeOS 25, Python 2 adds very little (~ 6% increase), but Avocado grows it by ~30%. With that in mind, the proposal for the JeOS 25 will include Python 2, but will *not* include Avocado itself. Feedback is welcome! [1] - While Avocado 47.0 has less requirements, I had to enable the "updates-testing" repo, so it actually increased the size of the image. Once the packages arrive on the main repo, there would probably be a decrease in size. [2] - https://github.com/avocado-framework/avocado-vt/pull/926 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Heads up: 7z based JeOS image to be deprecated and removed
Hi Avocado & Avocado-VT users, The following change was recently introduced in Avocado-VT: https://github.com/avocado-framework/avocado-vt/commit/43b18dbc1658a66558413d5a19436d8b17bcff70 Basically, it means that the default compression format for Avocado-VT image files changes from 7z to xz. We are still hosting the 7z based version of the JeOS 23 image, but that will be deprecated and removed *soon* (a few days). To avoid any impact, make sure you're running a version of Avocado-VT with those changes. This currently means the latest master or you own version with those changes cherry-picked. Please let me know of any breakage, ideally *before* the removal of the 7z based image. Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Heads up: 7z based JeOS image to be deprecated and removed
FIY, the JeOS image with 7z compression has now been removed from the server. On 03/18/2017 05:18 PM, Cleber Rosa wrote: > Hi Avocado & Avocado-VT users, > > The following change was recently introduced in Avocado-VT: > > https://github.com/avocado-framework/avocado-vt/commit/43b18dbc1658a66558413d5a19436d8b17bcff70 > > Basically, it means that the default compression format for Avocado-VT > image files changes from 7z to xz. We are still hosting the 7z based > version of the JeOS 23 image, but that will be deprecated and removed > *soon* (a few days). > > To avoid any impact, make sure you're running a version of Avocado-VT > with those changes. This currently means the latest master or you own > version with those changes cherry-picked. > > Please let me know of any breakage, ideally *before* the removal of the > 7z based image. > > Regards, > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] python-aexpect (drop aexpect requirement and package from repo)
On 03/24/2017 05:25 PM, Merlin Mathesius wrote: > > On 03/17/2017 02:05 PM, Merlin Mathesius wrote: >> On 03/08/2017 05:07 PM, Cleber Rosa wrote: >> >>>> Sure thing. I just submitted requests to publish the python-aexpect >>>> package for the following distros: >>>> >>>> * F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-94bb07357b >>>> * EL7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-5afaa778a4 >>>> >>>> Once the packages hit the testing repos, feel free to check them out and >>>> give karma. >>> Thanks a lot! I will give, buy and ask for karma donations! :) >>> >> FYI-- >> >> Despite a lack of karma donations, the F24 update was just pushed to >> stable. It should reach the mirrors soon. >> >> EL7 will have to wait another week before it can be pushed. > The EL7 update has now been pushed to stable. > > Merlin > Very good! Thanks Merlin! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 36.4 (LTS)
This is an announcement for the users of our Long Term Stability version of Avocado. This is a minor release that introduces bug fixes that are considered important to our users. Bugfixes * A fix for crashes on broken pipes * Avocado won't let broken terminals anymore (such as ones without echo). * Other backports of changes necessary to keep Avocado LTS up and running on newer development environments. For a full list of changes, please refer to: https://github.com/avocado-framework/avocado/compare/36.3...36.4 LTS in a nutshell = The LTS releases have a special cycle that lasts for 18 months. Avocado usage in production environments should favor the use of this LTS release, instead of non-LTS releases. For more information, please refer to: https://www.redhat.com/archives/avocado-devel/2016-May/msg00025.html https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html Install Avocado === Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#installing-avocado Updated RPM packages are available in the project repos for EPEL 6, EPEL 7 and Fedora 25[1]. Users subscribed to the LTS "channel", will get this 36.4 update, while users using the non-LTS repo, will currently fetch 47.0 after an update. Happy hacking and testing! [1] - An issue has been found with Fedora 24, possibly related to the version of the "fabric" package there. This prevents the build of Fedora 24 packages at the moment. After the issue is resolved, Fedora 24 packages of Avocado 36.4 will be made available. It's tracked at https://trello.com/c/vMDsztJg/979-fabric-avocado-lts-fedora-24-failure-to-build-packages . -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Pre/Post Mini-RFC
===
Pre/Post Mini-RFC
===
This is a status report on the various Pre/Post like interfaces we
have for Jobs and Tests.
The goal is to identify what's currently badly defined, and fix the
interface and implementations so that their goals are clearly
understandable. It should set the tone for other new pre/post
interfaces.
As we start to define a road map for a new LTS release, it's essential
to eliminate any source of confusion to users and to Avocado
contributors.
Pre, as in previous to its execution
So far, Avocado has implemented some "Pre" interfaces that are actually
executed by the subject itself. This is, by definition, very wrong.
An analogy may be useful. Suppose we are hosting a car race. Now
suppose we have a checklist to run *before* the race, such as checking
the conditions of the asphalt. One would never assume, rightfully so,
that this would be performed as *part of the race*, so no race cars
would be involved in checking the asphalt conditions. Instead, the
race organizers, would do that, before showing the competitors the
green light.
It's OK, though, for the race organizers to know about the race
conditions (duration, rules for pit stops, etc) and the cars, so that
the most adequate conditions are ensured.
If we were writing code that would map this analogy, we would have::
class Organizer(object):
def pre_race(self, race_conditions):
check_asphalt_conditions()
class Race(object):
def warmup_lap(self):
send_pace_car(speed=50)
It would be confusing, to have something like this::
class Race(object):
def pre_race(self):
prepare_a_race_within_a_race_is_wrong()
Or even::
class Race(object):
def pre(self):
"""
Pre what?
"""
prepare_a_err_dunno()
But it would be OK to have::
class Race(object):
def pre_pit_stop(self):
notify_mechanics()
Example with current Avocado
Avocado has a plugin interfaces called "JobPre". It's described as
"Base plugin interface for adding actions before a job **runs**".
This is clearly wrong, because it's executed as part of the
``avocado.core.Job.pre_tests()`` method
(https://github.com/avocado-framework/avocado/blob/47.0/avocado/core/job.py#L423)
::
def pre_tests(self):
...
self._job_pre_post_dispatcher.map_method('pre', self)
...
And it's even more incorrect because ``avocado.core.Job.pre_tests()``
is called as part of ``avocado.core.Job.run()``
(https://github.com/avocado-framework/avocado/blob/47.0/avocado/core/job.py#L476)
::
def run(self):
...
try:
self.create_test_suite()
self.pre_tests()
...
Now, **another** interface, called "JobPreTests" exists, and is
described as "Base plugin interface for adding actions before a job
runs tests", which is fine, and it gets executed at
``avocado.core.job.pre_tests()``.
Proposal
The proposal is to have "Pre" interfaces that are external to the
subjects.
This means that it's OK to have "JobPreTests", where a job does
something before a test (is executed). For further clarification,
such an interface could even be called "JobPreTestsExecution", but it
seems to add verbosity without adding further value. Other examples
may actually benefit from being more precise.
With regards to "JobPre", it is a valid requirement, but should
never be implemented within the Job. A "JobPre", is, in reality, a
"JobInit" or "JobSetUp", and should be named as such. Just think
of a "job initializing **itself**", or "setting up **itself**".
If the Avocado command line application creates a Job, then it should
be the one calling a "JobPre" implementations, with enough information
it has about the upcoming job.
Right now, it may be difficult to do this in Avocado, as a lot of the
configuration about a Job exists solely within an already instantiated
Job. But this is actually a good thing, because the separation of
configuration coming from sources such as files, command line,
parameter system and the Job itself is one of the requirements of the
much talked about "Job API".
For users of the Job API, utility methods could exist, but would still
require actions/code such as the following pseudo-code::
from avocado import Job
from avocado import load_default_config
from plugins.pre_job import run_pre_job
load_default_config()
run_pre_job()
job = Job()
run_post_job()
For a "JobInit" or "JobSetUp" interface, they could be automatically
called before a Job is considered ready to "run()".
References
==
This (mini) RFC is intended to shape the directi
[Avocado-devel] Sprint #48 Release and #49 Planning Meeting
Dear Avocado users and developers, I'd like to invite you all to a sprint release and planning meeting. It is going to take place at `date -d "Mon Apr 3 10:00:00 EDT 2017"` using the Hangouts on Air. https://www.youtube.com/watch?v=Wnh3odoph1M You can decide whether you want to join directly, or just watch the streaming and ask questions via IRC irc://irc.oftc.net/#avocado The meeting will be split in two parts (roughly 30 min each): * Part 1: Sprint Review * Review of the changes introduced during this sprint * Short demonstrations of some of the new features * Live sprint status: release readiness, last minute blockers, etc. * Part 2: Sprint Planning * New sprint planning boards will be created and its tasks prioritized. * Quick review of the expectations for the next sprint (high-level vision, goals, highlights) The meeting is going to be recorded and shared on Youtube afterwards. It is worth mentioning that most of the tasks during the meeting are going to take place on our Trello board: https://trello.com/b/WbqPNl2S/avocado Best Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Pre-release 48.0 test plan results - FAIL (Avocado-Virt issues)
Test Plan: Release Test Plan Run by 'cleber' at 2017-04-03T06:29:28.161571 PASS: 'Avocado source is sound': PASS: 'Avocado source does not contain spelling errors': PASS: 'Avocado RPM build': PASS: 'Avocado RPM lint': PASS: 'Avocado RPM install': PASS: 'Avocado Test Run on RPM based installation': PASS: 'Avocado Test Run on Virtual Machine': PASS: 'Avocado Test Run on Remote Machine': PASS: 'Avocado Remote Machine HTML report': PASS: 'Avocado Server Source Checkout and Unittests': PASS: 'Avocado Server Run': PASS: 'Avocado Server Functional Test': PASS: 'Avocado Virt and VT Source Checkout': FAIL: 'Avocado Virt Bootstrap': The JeOS change has to be replicated to Avocado-Virt: Failed to get SHA1 from file: HTTP Error 404: Not Found PASS: 'Avocado Virt Boot Test Run and HTML report': PASS: 'Avocado Virt - Assignment of values from the cmdline': FAIL: 'Avocado Virt - Migration test': ERROR 1-avocado-virt-tests/qemu/migration/migration.py:MigrationTest.test_migrate -> TypeError: 'NoneType' object has no attribute '__getitem__' PASS: 'Avocado VT': PASS: 'Avocado HTML report sysinfo': PASS: 'Avocado HTML report links': PASS: 'Paginator': avocado: dfcdc8b64a3b0add8d9b7f91e75b0483222c54d6 avocado-server: 8cec494edcf160a8bc9f61e174ff7bcedb1e6800 avocado-virt: 94712d5664cfe85f4fbbd60cac36c9bf42308edb avocado-virt-tests: cc185f45cdadef0898f6a382ae81179cdfa1e5df avocado-vt: 78941a7e1eba8ff7a3ccc5af5995b24ef1d3a78e signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Pre-release 48.0 test plan results - FAIL (Avocado-Virt issues)
On 04/03/2017 07:19 AM, Cleber Rosa wrote: > Test Plan: Release Test Plan > Run by 'cleber' at 2017-04-03T06:29:28.161571 > > PASS: 'Avocado source is sound': > PASS: 'Avocado source does not contain spelling errors': > PASS: 'Avocado RPM build': > PASS: 'Avocado RPM lint': > PASS: 'Avocado RPM install': > PASS: 'Avocado Test Run on RPM based installation': > PASS: 'Avocado Test Run on Virtual Machine': > PASS: 'Avocado Test Run on Remote Machine': > PASS: 'Avocado Remote Machine HTML report': > PASS: 'Avocado Server Source Checkout and Unittests': > PASS: 'Avocado Server Run': > PASS: 'Avocado Server Functional Test': > PASS: 'Avocado Virt and VT Source Checkout': > FAIL: 'Avocado Virt Bootstrap': The JeOS change has to be replicated to > Avocado-Virt: Failed to get SHA1 from file: HTTP Error 404: Not Found ^ PR #97 fixes this: https://github.com/avocado-framework/avocado-virt/pull/97 > PASS: 'Avocado Virt Boot Test Run and HTML report': > PASS: 'Avocado Virt - Assignment of values from the cmdline': > FAIL: 'Avocado Virt - Migration test': ERROR > 1-avocado-virt-tests/qemu/migration/migration.py:MigrationTest.test_migrate > -> TypeError: 'NoneType' object has no attribute '__getitem__' ^ This was tested, and failed, on a machine with nested virt. On a physical machine the test passed. Job log can be seen here: https://paste.fedoraproject.org/paste/bgYfkLxsHoXRE1KEKTKLBV5M1UNdIGYhyRLivL9gydE=/ Either way this is worth investigating, because it could be a QEMU/KVM migration or migrate.py issue. > PASS: 'Avocado VT': > PASS: 'Avocado HTML report sysinfo': > PASS: 'Avocado HTML report links': > PASS: 'Paginator': > > avocado: dfcdc8b64a3b0add8d9b7f91e75b0483222c54d6 > avocado-server: 8cec494edcf160a8bc9f61e174ff7bcedb1e6800 > avocado-virt: 94712d5664cfe85f4fbbd60cac36c9bf42308edb > avocado-virt-tests: cc185f45cdadef0898f6a382ae81179cdfa1e5df > avocado-vt: 78941a7e1eba8ff7a3ccc5af5995b24ef1d3a78e > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Multi Stream Test Support
1 %s" % self.streams[1].host))
self.streams.wait()
self.assertTrue(self.streams[1:2].success)
self.assertFalse(self.streams[3:4].success)
Support for synchronized execution also maps really well to the
slicing example. For instance, consider this::
from avocado import Test
from avocado.streams.code import shell
class InterCommunication(Test):
def test(self):
self.streams[1].run(shell("ping -c 60 %s" % self.streams[2].host)
self.streams[2].run(shell("ping -c 60 %s" % self.streams[1].host))
ddos = shell("ddos --target %s" self.streams[1].host)
self.streams[3:4].run(ddos, synchronized=True)
self.streams[1:2].wait()
self.assertTrue(self.streams.success)
This instructs streams 1 and 2 to start connectivity checks as soon as
they **individually** can, while, for a full DDOS effect, streams 3
and 4 would start only when they are both ready to do so.
Feedback and future versions
This being an RFC, feedback is extremely welcome. Also, exepect new
versions
based on feedback, discussions and further development of the ideas
initially
exposed here.
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
signature.asc
Description: OpenPGP digital signature
Re: [Avocado-devel] Sprint #48 Release and #49 Planning Meeting
On 03/31/2017 08:58 AM, Cleber Rosa wrote: > Dear Avocado users and developers, > > I'd like to invite you all to a sprint release and planning meeting. > It is going to take place at `date -d "Mon Apr 3 10:00:00 EDT 2017"` > using the Hangouts on Air. > > https://www.youtube.com/watch?v=Wnh3odoph1M > Hangouts URL is: https://hangouts.google.com/hangouts/_/3ng4l6d435gv5hjfzoactypqhme > You can decide whether you want to join directly, or just watch the > streaming and ask questions via IRC irc://irc.oftc.net/#avocado > > The meeting will be split in two parts (roughly 30 min each): > > * Part 1: Sprint Review > * Review of the changes introduced during this sprint > * Short demonstrations of some of the new features > * Live sprint status: release readiness, last minute blockers, etc. > > * Part 2: Sprint Planning > * New sprint planning boards will be created and its tasks prioritized. > * Quick review of the expectations for the next sprint (high-level > vision, goals, highlights) > > The meeting is going to be recorded and shared on Youtube afterwards. > It is worth mentioning that most of the tasks during the meeting are > going to take place on our Trello board: > > https://trello.com/b/WbqPNl2S/avocado > > Best Regards, > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 48.0: Lost Boundaries
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/48.0/release_notes/48_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/48.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Promote XuHan as avocado-vt maintainer
On 04/04/2017 11:30 PM, xutian wrote: > Add XuHan and my manager Michen in CC list. > > > On 04/05/2017 11:25 AM, xutian wrote: >> Hi list, >> >> I will leave Red Hat on May 1 to work elsewhere, and in the future I >> may not have too much time to carry out the maintenance work of >> avocado-vt, and I recommend my colleague Han Xu to take over me. >> >> Thanks, >> >> Xu >> > Hi Xu, Sad to see you leaving, because it's an obvious (at least partial) loss for Avocado-VT. Thanks for all of your hard work on it! And, based on your suggestion, I'm promoting Xu Han to the maintainers group. Best of luck on your next endeavors! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Tips for making a standalone test script Avocado-friendly?
ml#writing-a-simple-test [3] - https://github.com/avocado-framework/avocado/blob/master/examples/tests/simplewarning.sh [4] - http://avocado-framework.readthedocs.io/en/48.0/WritingTests.html#test-statuses [5] - http://avocado-framework.readthedocs.io/en/48.0/WritingTests.html#environment-variables-for-simple-tests [6] - https://github.com/avocado-framework/avocado/blob/master/optional_plugins/robot/avocado_robot/__init__.py#L79 [7] - https://github.com/avocado-framework/avocado/blob/master/optional_plugins/robot/avocado_robot/__init__.py#L52 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Multi Stream Test Support
On 04/05/2017 03:29 AM, Lukáš Doktor wrote: > Dne 3.4.2017 v 15:48 Cleber Rosa napsal(a): >> Note: this document can be view in rendered format at: >> >> https://github.com/clebergnu/avocado/blob/RFC_multi_stream_v1/docs/source/rfcs/multi_stream.rst >> >> >> === >> Multi Stream Test Support >> === >> >> Introduction >> >> >> Avocado currently does not provide test writers with standard tools >> or guidelines for developing tests that spawn multiple machines. >> >> Since these days the concept of a "machine" is blurring really >> quickly, this proposal for Avocado's version of "multi machine" test >> support is more abstract (that's an early and quick explanation of >> what a "stream" means). One of the major goal is to be more flexible >> and stand the "test" (pun intended) of time. >> >> This is a counter proposal to a previous RFC posted and discussed on >> Avocado development mailing list. Many of the concepts detailed here >> were introduced there: >> >> * https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html >> * https://www.redhat.com/archives/avocado-devel/2016-March/msg00035.html >> * https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html >> * https://www.redhat.com/archives/avocado-devel/2016-April/msg00072.html >> >> Background >> == >> >> The prior art that influences Avocado the most is Autotest. The >> reason is that many of the Avocado developers worked on Autotest >> before, and both share various common goals. Let's use Autotest, >> which provided support for multiple machine test support as a basis >> for comparison. >> >> Back in the Autotest days, a test that would spawn multiple machines >> was a very particular type of test. To write such a test, one would >> write a **different** type of "control file" (a server one). Then, by >> running a "server control file" with an **also different** command >> line application (``autoserv``, A.K.A. ``autotest-remote``), the >> server control file would have access to some special variables, such >> as the ``machines`` one. By using an **also different** type of job >> implementation, the control file could run a given **Python function** >> on these various ``machines``. >> >> An actual sample server control file (``server/samples/reboot.srv``) >> for Autotest looks like this:: >> >>1 def run(machine): >>2 host = hosts.create_host(machine) >>3 host.reboot() >>4 >>5 job.parallel_simple(run, machines) >> >> Line #5 makes use of the different (server) job implementation to run >> function ``run`` (defined in line #1) in parallel on machines given by >> the special variable ``machines`` (made available by the also special >> ``autoserv`` tool). >> >> This quick background check shows two important facts: >> >> 1) The functionality is not scoped to tests. It's not easy to understand >>where a test begins or ends by looking at such a control file. >> >> 2) Users (and most importantly test writers) have to learn about >>different tools and APIs when writing "multi machine" code; >> >> 3) The machines are defined outside the test itself (in the form of >>arguments to the ``autoserv`` command line application); >> >> Please keep these Autotest characteristics in mind: Avocado's multi >> stream test support goals will be presented shortly, and will detail >> how they contrast with those. >> >> Avocado's Multi Stream Test Support Goals >> = >> >> This is a hopefully complete summary of our goals: >> >> 1) To not require a different type of test, that is, allow users >>to *write* a plain `avocado.Test` while still having access to >>multi stream goodies; >> >> 2) To allow for clear separation between the test itself and its >>execution environment (focus here on the execution streams >>environment); >> >> 3) To allow increased flexibility by abstracting the "machines" >>concept into "excution streams"; >> >> 4) To allow for even increased flexibility by allowing test writers to >>use not only Python functions, but other representations of code to >>be executed on those separate streams; >> >> Comparison with prior art >> --
[Avocado-devel] Avocado "assets" and "repos" server scheduled downtime
Hi everyone, The Avocado project hosts two services: * repos: hosts official packages for Avocado and Avocado-VT, used when you install avocado or avocado-vt from the official repos * assets: hosts files such as the official JeOS images, used when you run `avocado vt-bootstrap` or `avocado virt-bootstrap` These will be offline next Saturday, April the 8th, starting from 8:00 AM EDT. They will be given more computing, storage and network resources, to address some reliability issues reported by some users. The downtime should not take longer than 2 hours, but a message will be sent to the mailing when the maintenance period is over. Thanks! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Tips for making a standalone test script Avocado-friendly?
On 04/06/2017 11:00 AM, Lukáš Doktor wrote: > Dne 5.4.2017 v 17:06 Cleber Rosa napsal(a): >> >> On 04/05/2017 09:00 AM, Eduardo Habkost wrote: >>> >>> Hi, >>> >>> I have been writing a few standalone Python scripts[1] to test >>> QEMU recently, and I would like to make them more useful for >>> people running tests using Avocado. >>> >>> Most of them work this way: >>> 1) Query QEMU to check which >>>architectures/machine-types/CPU-models/devices/options >>>it supports >>> 2) Run QEMU multiple times for each >>>architectures/machine-types/CPU-models/devices/options >>>combination I want to test >>> 3) Report success/failure/skip results (sometimes including >>>warnings) for each combination >>> >> >> I went ahead and tried one of them: >> >> QTEST_QEMU_BINARY=../x86_64-softmmu/qemu-system-x86_64 python >> query-cpu-model-test.py >> .. >> -- >> Ran 2 tests in 34.227s >> >> OK >> >> One very important aspect here is result granularity. You're currently >> using Python's unittest.TestCase, which in this case gives you a >> granularity of two tests. At this point, Avocado could find those two >> tests (with some help[1]): >> >> $ ~/src/avocado/avocado/contrib/scripts/avocado-find-unittests >> query-cpu-model-test.py >> query-cpu-model-test.CPUModelTest.testTCGModels >> query-cpu-model-test.CPUModelTest.testKVMModels >> >> The other granularity level that can be achieved here is a test per >> executed script: >> >> $ QTEST_QEMU_BINARY=../x86_64-softmmu/qemu-system-x86_64 avocado run >> query-cpu-model-test.py >> JOB ID : 11f0a5fbb02e6eb67580dd33270867b039806585 >> JOB LOG: >> /home/cleber/avocado/job-results/job-2017-04-05T10.31-11f0a5f/job.log >> (1/1) query-cpu-model-test.py: PASS (33.22 s) >> RESULTS: PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | >> CANCEL 0 >> TESTS TIME : 33.22 s >> JOB HTML : >> /home/cleber/avocado/job-results/job-2017-04-05T10.31-11f0a5f/html/results.html >> >> >> >> Which is nothing but our SIMPLE[2] tests support. >> >> About reporting results, PASS, FAIL and WARN are available for SIMPLE >> tests. PASS is exit status 0, FAIL is anything but 0, and WARN is a PASS >> that looks for a pattern in the test output[3][4]. >> >>> I would like to keep the test scripts easy to run without >>> installing extra dependencies, so I want them to keep working as >>> standalone scripts even if Avocado modules aren't available. >> >> That's OK, but I would like to add a few notes: >> >> 1) When removing the unittest dependency, you'll most probably end up >> creating custom mechanisms for locating and executing test entry points >> and reporting those results. >> >> 2) We've worked to make sure Avocado is really easy to depend on. For >> instance, it's available on stock Fedora (package name is >> python-avocado) and on PyPI (installing it just requires a `pip install >> avocado-framework`). >> >>> Adding a few "if avocado_available:" lines to the script would be >>> OK, though. >>> >> >> I can't really recommend this optional check of Avocado for INSTRUMENTED >> tests. I really think the complexity added would bring more bad than >> good. >> >> If your tests are going to be treated as SIMPLE tests, you could just >> make a check for environment variables that Avocado makes available[5]. >> >>> Do you have any suggestions for making the test result output >>> from those scripts easily consumable by the Avocado test runner? >>> >> >> If those tests are treated as SIMPLE tests, then all of their output >> will automatically be added to the Avocado logs. >> >> Now, I do understand what your hopes are (or were after my possibly >> disappointing recommendations :). Ideally, Avocado would be able to >> find the tests available on your standalone scripts[6], would know how >> to run the individual tests[7]. Right now, this requires writing a >> plugin. I can see some ideas for a "Python universal" plugin here, that >> would find other hints of tests in plain Python source code. If you >> think that makes sense, let's discuss it further. >> > Actually we already have `robot` plugin which is able to discover >
Re: [Avocado-devel] Multi Stream Test Support
On 04/06/2017 05:01 AM, Lukáš Doktor wrote: > Dne 5.4.2017 v 18:13 Cleber Rosa napsal(a): >> >> >> On 04/05/2017 03:29 AM, Lukáš Doktor wrote: >>> Dne 3.4.2017 v 15:48 Cleber Rosa napsal(a): >>>> Note: this document can be view in rendered format at: >>>> >>>> https://github.com/clebergnu/avocado/blob/RFC_multi_stream_v1/docs/source/rfcs/multi_stream.rst >>>> >>>> >>>> >>>> === >>>> Multi Stream Test Support >>>> === >>>> >>>> Introduction >>>> >>>> >>>> Avocado currently does not provide test writers with standard tools >>>> or guidelines for developing tests that spawn multiple machines. >>>> >>>> Since these days the concept of a "machine" is blurring really >>>> quickly, this proposal for Avocado's version of "multi machine" test >>>> support is more abstract (that's an early and quick explanation of >>>> what a "stream" means). One of the major goal is to be more flexible >>>> and stand the "test" (pun intended) of time. >>>> >>>> This is a counter proposal to a previous RFC posted and discussed on >>>> Avocado development mailing list. Many of the concepts detailed here >>>> were introduced there: >>>> >>>> * >>>> https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html >>>> * >>>> https://www.redhat.com/archives/avocado-devel/2016-March/msg00035.html >>>> * >>>> https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html >>>> * >>>> https://www.redhat.com/archives/avocado-devel/2016-April/msg00072.html >>>> >>>> Background >>>> == >>>> >>>> The prior art that influences Avocado the most is Autotest. The >>>> reason is that many of the Avocado developers worked on Autotest >>>> before, and both share various common goals. Let's use Autotest, >>>> which provided support for multiple machine test support as a basis >>>> for comparison. >>>> >>>> Back in the Autotest days, a test that would spawn multiple machines >>>> was a very particular type of test. To write such a test, one would >>>> write a **different** type of "control file" (a server one). Then, by >>>> running a "server control file" with an **also different** command >>>> line application (``autoserv``, A.K.A. ``autotest-remote``), the >>>> server control file would have access to some special variables, such >>>> as the ``machines`` one. By using an **also different** type of job >>>> implementation, the control file could run a given **Python function** >>>> on these various ``machines``. >>>> >>>> An actual sample server control file (``server/samples/reboot.srv``) >>>> for Autotest looks like this:: >>>> >>>>1 def run(machine): >>>>2 host = hosts.create_host(machine) >>>>3 host.reboot() >>>>4 >>>>5 job.parallel_simple(run, machines) >>>> >>>> Line #5 makes use of the different (server) job implementation to run >>>> function ``run`` (defined in line #1) in parallel on machines given by >>>> the special variable ``machines`` (made available by the also special >>>> ``autoserv`` tool). >>>> >>>> This quick background check shows two important facts: >>>> >>>> 1) The functionality is not scoped to tests. It's not easy to >>>> understand >>>>where a test begins or ends by looking at such a control file. >>>> >>>> 2) Users (and most importantly test writers) have to learn about >>>>different tools and APIs when writing "multi machine" code; >>>> >>>> 3) The machines are defined outside the test itself (in the form of >>>>arguments to the ``autoserv`` command line application); >>>> >>>> Please keep these Autotest characteristics in mind: Avocado's multi >>>> stream test support goals will be presented shortly, and will detail >>>> how they contrast with those. >>>> >>>> Avocado's Multi Stream Test Support Goals >>>>
Re: [Avocado-devel] Tips for making a standalone test script Avocado-friendly?
On 04/06/2017 02:20 PM, Eduardo Habkost wrote: > On Thu, Apr 06, 2017 at 12:38:31PM -0400, Cleber Rosa wrote: > [...] >>>>> Do you have any suggestions for making the test result output >>>>> from those scripts easily consumable by the Avocado test runner? >>>>> >>>> >>>> If those tests are treated as SIMPLE tests, then all of their output >>>> will automatically be added to the Avocado logs. >>>> >>>> Now, I do understand what your hopes are (or were after my possibly >>>> disappointing recommendations :). Ideally, Avocado would be able to >>>> find the tests available on your standalone scripts[6], would know how >>>> to run the individual tests[7]. Right now, this requires writing a >>>> plugin. I can see some ideas for a "Python universal" plugin here, that >>>> would find other hints of tests in plain Python source code. If you >>>> think that makes sense, let's discuss it further. >>>> >>> Actually we already have `robot` plugin which is able to discover >>> `robot`-like tests. How about creating a `unittest` loader, which would >>> do the `avocado-find-unittest` and discover them individually? By >>> default nothing would change as the `.py` files would be recognized as >>> instrumented/simple tests, but when you change the order of loaders it'd >>> detect them as unittests: >>> >> >> Eduardo mentioned that he'd eventually get rid of the unittest >> dependency. That's why I was suggesting/brainstorming about an even >> more generic way of providing hints that there are tests on a Python >> source file. > > I planned to get rid of the unittest dependency because it did > not seem useful to me. But if it provides Avocado test discovery > ability for free, I would probably keeping it. > OK, so that let's keep track of that: https://trello.com/c/j9IE7BHy/992-loader-add-native-python-unittest-support > Now, another problem is that I wish a test granuality that > requires running QEMU at least once during the test discovery > phase (e.g. one test case for each machine-type/device/CPU). > Maybe this will break some assumptions in Avocado? > That would be possible, again, with a custom plugin. The Avocado-VT loader, for instance, does "what-not" during test discovery. How to do that in a generic (and requirements free) way, may be a little more tricky (or interesting). One way of possibly doing this (which is halfway there, but still not supported), is to use the `avocado.core.Job` API, which we intend to make public in the future. You could still have your standlone scripts (added simplicity here if they're already unittest.TestCase based), and write a custom Job that would create a custom test suite: ... # regular script content ... if __name__ == '__main__': if under_avocado(): # fictitious function from avocado import Job class QemuDynamicJob(Job): def create_test_suite(self): # [1] methods = find_tests_on_this_file() self.test_suite = methods * get_cpu_models() # [2] QemuDynamicJob().run() > Are tests able to get a few input parameters during thest > discovery? e.g. I want to tell the test script/module the list of > QEMU binaries I want to test. Today I use an environment variable > or command-line parameter for that. > Right now a custom loader can have access to all sorts of parameters. Tests do not discover themselves, so that would be against the current architecture, but jobs do, as explained before. Let me know if that makes sense. [1] - http://avocado-framework.readthedocs.io/en/48.0/api/core/avocado.core.html#avocado.core.job.Job.create_test_suite [2] - http://avocado-framework.readthedocs.io/en/48.0/api/core/avocado.core.html#avocado.core.job.Job.test_suite -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Avocado "assets" and "repos" server scheduled downtime
The servers are now back online. On 04/05/2017 04:18 PM, Cleber Rosa wrote: > Hi everyone, > > The Avocado project hosts two services: > > * repos: hosts official packages for Avocado and Avocado-VT, used when > you install avocado or avocado-vt from the official repos > > * assets: hosts files such as the official JeOS images, used when you > run `avocado vt-bootstrap` or `avocado virt-bootstrap` > > These will be offline next Saturday, April the 8th, starting from 8:00 > AM EDT. They will be given more computing, storage and network > resources, to address some reliability issues reported by some users. > > The downtime should not take longer than 2 hours, but a message will be > sent to the mailing when the maintenance period is over. > > Thanks! > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Multi Stream Test Support
On 04/10/2017 01:56 PM, Jeff Nelson wrote: > On Mon, 3 Apr 2017 09:48:16 -0400 > Cleber Rosa wrote: > >> Note: this document can be view in rendered format at: >> >> https://github.com/clebergnu/avocado/blob/RFC_multi_stream_v1/docs/source/rfcs/multi_stream.rst >> > > Here's my feedback. Please let me know if you have any questions. > > >> === >> Multi Stream Test Support >> === >> >> Introduction >> >> >> Avocado currently does not provide test writers with standard tools >> or guidelines for developing tests that spawn multiple machines. >> > > > s/spawn multiple machines/require multiple, cooperating execution > contexts/ ? > > The reason for using "multiple machine" here is that, up to the this point, that's how we have seen users referring to this particular feature. I felt that an introductory line without such reference would fail to establish a connection to it. Still, "multiple execution contexts" sounds good, and I'll try to incorporate both ideas, if at all possible, in the next version. >> Since these days the concept of a "machine" is blurring really >> quickly, this proposal for Avocado's version of "multi machine" test >> support is more abstract (that's an early and quick explanation of >> what a "stream" means). One of the major goal is to be more flexible >> and stand the "test" (pun intended) of time. > > This introduction mentions "machine" multiple times. I found this > puzzling, because the proposal is titled "multi stream test support". > The machine view is an old, historical idea and I know next to nothing > about it, so it's not very helpful to me. > Right, you're spot on here. I say that because "multiple machine" may mean *a lot* to users with previous involvement and engagement in similar tools. > I suggest writing an intro for a target audience that knows nothing > about the past but is familiar with how avocado operates today. The > intro should describe the goal or objective of the design. Reference to > older implementations and ideas can be done in the Background section. > > OK, makes sense. >> This is a counter proposal to a previous RFC posted and discussed on >> Avocado development mailing list. Many of the concepts detailed here >> were introduced there: >> >> * https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html >> * https://www.redhat.com/archives/avocado-devel/2016-March/msg00035.html >> * https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html >> * https://www.redhat.com/archives/avocado-devel/2016-April/msg00072.html > > Are there other related RFCs that this proposal affects? For example, > is there an RFC that describes the avocado callable test interface? If > so, referencing it might be useful for readers. > > I don't think so. The implementation of the ideas given here should not affect the concepts that are already consolidated in Avocado. A current Avocado user hoping to execute his/her tests, should not be affected in any way, shape or form, by upgrading to a version that contains the implementation of the ideas proposed here. >> Background >> == >> >> The prior art that influences Avocado the most is Autotest. The >> reason is that many of the Avocado developers worked on Autotest >> before, and both share various common goals. Let's use Autotest, >> which provided support for multiple machine test support as a basis >> for comparison. >> >> Back in the Autotest days, a test that would spawn multiple machines >> was a very particular type of test. To write such a test, one would >> write a **different** type of "control file" (a server one). Then, by >> running a "server control file" with an **also different** command >> line application (``autoserv``, A.K.A. ``autotest-remote``), the >> server control file would have access to some special variables, such >> as the ``machines`` one. By using an **also different** type of job >> implementation, the control file could run a given **Python function** >> on these various ``machines``. >> >> An actual sample server control file (``server/samples/reboot.srv``) >> for Autotest looks like this:: >> >>1 def run(machine): >>2 host = hosts.create_host(machine) >>3 host.reboot() >>4 >>5 job.parallel_simple(run, machines) >> >> Line #5 makes use of the different (serve
[Avocado-devel] [RFC v3]: Avocado maintainability and integration with avocado-vt
s) ... avocado-vt -- avocado-vt is an Avocado plugin that allows "VT tests" to be run inside Avocado. It's a third-party project maintained mostly by Engineers from Red Hat QE with assistance from the Avocado team and other community members. It's a general consensus that QE teams use avocado-vt directly from git, usually following the master branch, which they control. There's no official maintenance or stability statement for avocado-vt. Even though the upstream community is quite friendly and open to both contributions and bug reports, avocado-vt is made available without any promises for compatibility or supportability. When packaged and versioned, avocado-vt rpms should be considered just snapshots, available in packaged form as a convenience to users outside of the avocado-vt development community. Again, they are made available without any promises of compatibility or stability. * Which Avocado version should be used by avocado-vt? This is up to the avocado-vt community to decide, but the current consensus is that to guarantee some stability in production environments, avocado-vt should stick to a specific LTS release of Avocado. In other words, the Avocado team recommends production users of avocado-vt not to install Avocado from its master branch or upgrade it from Sprint Releases. Given each LTS release will be maintained for 18 months, it should be reasonable to expect avocado-vt to upgrade to a new LTS release once a year or so. This process will be done with support from the Avocado team to avoid disruptions, with proper coordination via the avocado mailing lists. In practice the Avocado development team will keep watching avocado-vt to detect and document incompatibilities, so when the time comes to do an upgrade in production, it's expected that it should happen smoothly. * Will it be possible to use the latest Avocado and avocado-vt together? Users are welcome to *try* this combination. The Avocado development team itself will do it internally as a way to monitor incompatibilities and regressions. Whenever Avocado is released, a matching versioned snapshot of avocado-vt will be made. Packages containing those avocado-vt snapshots, for convenience only, will be made available in the regular Avocado repository. --- References: [1] - https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html [2] - http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#fedora-from-avocado-s-own-repo [3] - https://pypi.python.org/pypi -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 49.0: The Physician
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/49.0/release_notes/49_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/49.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] [RFC v3]: Avocado maintainability and integration with avocado-vt
On 04/20/2017 07:06 AM, Andrei Stepanov wrote: > Hello. > > Sounds good. > I would add a small note. > QE teams tend to run their tests against old RHEL (7.0/7.1/7.3) > versions as well. > Maybe this somehow correct your roadmap. > My point is to stay away from useless dependencies. > Ideally Avocado should run on RHEL7.0+official repo / RHEL 7.1 + > official repo / etc > Problem with EPEL is: EPEL can be installed only on the latest .z stream. > For example, if you have: EPEL + RHEL7.0 this is no more RHEL 7.0. > Actually you cannot update RHEL7.0 with EPEL. > It must be the latest RHEL7.x.z. > Now we use python-virtenv+pypi to overcome this. > Hi Andrei, This is a very valid point indeed. We've made things a little better by moving a lot of the optional plugins outside the base package. AFAIK, the base (python-)avocado package on EL7 requires EPEL only because of (python-)stevedore. I believe this can only be addressed by requesting this extra package to be added to EL7 (even if it's only on newer releases). But this actually brings a question: is EL7 + Avocado RPM from official repo, still RHEL7? I mean, from a QE perspective, is this acceptable? If not, the only real solution I see is also adding Avocado to EL7. Thanks, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] [RFC v3]: Avocado maintainability and integration with avocado-vt
On 04/25/2017 12:05 PM, Lukáš Doktor wrote:
> Dne 20.4.2017 v 03:42 Cleber Rosa napsal(a):
>> Hi Folks.
>>
> Hello Cleber,
>
> thank you for the updates, in general it's good, I have few minor
> suggestions in-line.
>
>> This RFC contains proposals and clarifications regarding the
>> maintenance and release processes of Avocado.
>>
>> We understand there are multiple teams currently depending on the
>> stability of Avocado and we don't want their work to be disrupted by
>> incompatibilities nor instabilities in new releases.
>>
>> This version is a minor update to RFC version 2[1], which drove the
>> release of Avocado 36.0 LTS. The Avocado team has plans for a new LTS
>> release in the near future, so please consider reading and providing
>> feedback on the proposals here.
>>
>> TL;DR:
>>
>> We plan to keep the current approach of sprint releases every 3-4
>> weeks, but we're introducing "Long Term Stability" releases which
>> should be adopted in production environments where users can't keep
>> up with frequent upgrades.
>>
>> Changes from v2:
>> - Wording changes on second paragraph ("... nor instabilities...")
>> - Clarified on "Introduction" that change of behavior is introduced
>> between regular releases
>> - Updated distro versions for which official packages are built
>> - Add more clear explanation on official packages on the various
>> hardware platforms
>> - Used more recent version numbers as examples, and the planned
>> new LTS version too
>> - Explain how users can get the LTS version when using tools such as
>> pip
>> - Simplified the timeline example, with examples that will possibly
>> match the future versions and releases
>> - Documented current status of avocado-vt releases and packages
>>
>> Changes from v1:
>> - Changed "Support" to "Stability" and "supported" to "maintained"
>> [Jeff Nelson]
>> - Misc improvements and clarifications in the
>> supportability/stability statements [Jeff Nelson, me]
>> - Fixed a few typos [Jeff Nelson, me]
>>
>>
>> Introduction
>> --
>>
>> We make new releases of Avocado every 3-4 weeks on average. In theory
>> at least, we're very careful with backwards compatibility. We test
>> Avocado for regressions and we try to document any issues, so
>> upgrading to a new version should be (again, in theory) safe.
>>
>> But in practice both intended and unintended changes are introduced
>> during development, and both can be frustrating for conservative
>> users. We also understand it's not feasible for users to upgrade
>> Avocado very frequently in a production environment.
>>
>> The objective of this RFC is to clarify our maintenance practices and
>> introduce Long Term Stability (LTS) releases, which are intended to
>> solve, or at least mitigate, these problems.
>>
>>
>> Our definition of maintained, or stable
>> ---
>>
>> First of all, Avocado and its sub-projects are provided 'AS IS' and
>> WITHOUT ANY WARRANTY, as described in the LICENSE file.
>>
>> The process described here doesn't imply any commitments or
>> promises. It's just a set of best practices and recommendations.
>>
>> When something is identified as "stable" or "maintained", it means the
>> development community makes a conscious effort to keep it working and
>> consider reports of bugs and issues as high priorities. Fixes
>> submitted for these issues will also be considered high priorities,
>> although they will be accepted only if they pass the general
>> acceptance criteria for new contributions (design, quality,
>> documentation, testing, etc), at the development team discretion.
>>
>>
>> Maintained projects and platforms
>> -
>>
>> The only maintained project as of today is the Avocado Test Runner,
>> including its APIs and core plugins (the contents of the main avocado
>> git repository).
>>
>> Other projects kept under the "Avocado Umbrella" in github may be
>> maintained by different teams (e.g.: avocado-vt) or be considered
>> experimental (e.g.: avocado-server and avocado-virt).
>>
>> More about avocado-vt in its own section further down.
>>
>> As a general rule, fixes and bug reports for Avocado when running in
>
[Avocado-devel] [RFC v4]: Long Term Stability (was Avocado maintainability and integration with avocado-vt)
release 52.0 * **LTS** release * 52lts branch is created * packages go into LTS repo * both **36.x LTS** and **52.x LTS** maintained from this point on 53.0 * sprint release * bug that also affects 52.0 is found, fix gets added to master and 52lts branches 54.0 * sprint release 54.0 **AND** LTS 52.1 release * minor bug that also affects 52.0 is found, fix gets added to master and 52lts branches 55.0 * sprint release * critical bug that affects 52.1 *only* is found, fix gets added to 52lts and 52.2 LTS is immediately released 56.0 * sprint release 57.0 * sprint release 58.0 * sprint release 59.0 * sprint release * EOL for **36.x LTS** (18 months since the release of 36.0) Avocado-VT == Avocado-VT is an Avocado plugin that allows "VT tests" to be run inside Avocado. It's a third-party project maintained mostly by Engineers from Red Hat QE with assistance from the Avocado team and other community members. It's a general consensus that QE teams use Avocado-VT directly from git, usually following the master branch, which they control. There's no official maintenance or stability statement for Avocado-VT. Even though the upstream community is quite friendly and open to both contributions and bug reports, Avocado-VT is made available without any promises for compatibility or supportability. When packaged and versioned, Avocado-VT rpms should be considered just snapshots, available in packaged form as a convenience to users outside of the Avocado-VT development community. Again, they are made available without any promises of compatibility or stability. * Which Avocado version should be used by Avocado-VT? This is up to the Avocado-VT community to decide, but the current consensus is that to guarantee some stability in production environments, Avocado-VT should stick to a specific LTS release of Avocado. In other words, the Avocado team recommends production users of Avocado-VT not to install Avocado from its master branch or upgrade it from Sprint Releases. Given each LTS release will be maintained for 18 months, it should be reasonable to expect Avocado-VT to upgrade to a new LTS release once a year or so. This process will be done with support from the Avocado team to avoid disruptions, with proper coordination via the avocado mailing lists. In practice the Avocado development team will keep watching Avocado-VT to detect and document incompatibilities, so when the time comes to do an upgrade in production, it's expected that it should happen smoothly. * Will it be possible to use the latest Avocado and Avocado-VT together? Users are welcome to *try* this combination. The Avocado development team itself will do it internally as a way to monitor incompatibilities and regressions. Whenever Avocado is released, a matching versioned snapshot of Avocado-VT will be made. Packages containing those Avocado-VT snapshots, for convenience only, will be made available in the regular Avocado repository. .. _Version 1: https://www.redhat.com/archives/avocado-devel/2016-April/msg6.html .. _Version 2: https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html .. _Version 3: https://www.redhat.com/archives/avocado-devel/2017-April/msg00032.html .. _YUM/DNF repos: http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#fedora-from-avocado-s-own-repo .. _PyPI: https://pypi.python.org/pypi -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] [RFC v4]: Long Term Stability (was Avocado maintainability and integration with avocado-vt)
On 04/27/2017 04:04 AM, Amador Segundo wrote:
> On Thu, Apr 27, 2017 at 1:07 AM, Cleber Rosa wrote:
>> ==
>> RFC: Long Term Stability
>> ==
>>
>> This RFC contains proposals and clarifications regarding the
>> maintenance and release processes of Avocado.
>>
>> We understand there are multiple teams currently depending on the
>> stability of Avocado and we don't want their work to be disrupted by
>> incompatibilities nor instabilities in new releases.
>>
>> This version is a minor update to previous versions of the same RFC
>> (see `Changelog`_) which drove the release of Avocado 36.0 LTS. The
>> Avocado team has plans for a new LTS release in the near future, so
>> please consider reading and providing feedback on the proposals here.
>>
>> TL;DR
>> =
>>
>> We plan to keep the current approach of sprint releases every 3-4
>> weeks, but we're introducing "Long Term Stability" releases which
>> should be adopted in production environments where users can't keep up
>> with frequent upgrades.
>>
>> Changelog
>> =
>>
>> Changes from `Version 3`_:
>>
>> * Converted formatting to REStructuredText
>> * Replaced "me" mentions on version 1 changelog with proper name
>>(Ademar Reis)
>> * Renamed section "Misc Details" to `Deployment Details`_
>> * Renamed "avocado-vt" to "Avocado-VT"
>> * Start the timeline example with version 36.0
>> * Be explicit on timeline example that a minor bug did not generate
>>an immediate release
>>
>> Changes from `Version 2`_:
>>
>> * Wording changes on second paragraph ("... nor instabilities...")
>> * Clarified on "Introduction" that change of behavior is introduced
>>between regular releases
>> * Updated distro versions for which official packages are built
>> * Add more clear explanation on official packages on the various
>>hardware platforms
>> * Used more recent version numbers as examples, and the planned
>>new LTS version too
>> * Explain how users can get the LTS version when using tools such as
>>pip
>> * Simplified the timeline example, with examples that will possibly
>>match the future versions and releases
>> * Documented current status of Avocado-VT releases and packages
>>
>> Changes from `Version 1`_:
>>
>> * Changed "Support" to "Stability" and "supported" to "maintained"
>>[Jeff Nelson]
>> * Misc improvements and clarifications in the
>>supportability/stability statements [Jeff Nelson, Ademar Reis]
>> * Fixed a few typos [Jeff Nelson, Ademar Reis]
>>
>> Introduction
>>
>>
>> We make new releases of Avocado every 3-4 weeks on average. In theory
>> at least, we're very careful with backwards compatibility. We test
>> Avocado for regressions and we try to document any issues, so
>> upgrading to a new version should be (again, in theory) safe.
>>
>> But in practice both intended and unintended changes are introduced
>> during development, and both can be frustrating for conservative
>> users. We also understand it's not feasible for users to upgrade
>> Avocado very frequently in a production environment.
>>
>> The objective of this RFC is to clarify our maintenance practices and
>> introduce Long Term Stability (LTS) releases, which are intended to
>> solve, or at least mitigate, these problems.
>>
>>
>> Our definition of maintained, or stable
>> ===
>>
>> First of all, Avocado and its sub-projects are provided 'AS IS' and
>> WITHOUT ANY WARRANTY, as described in the LICENSE file.
>>
>> The process described here doesn't imply any commitments or
>> promises. It's just a set of best practices and recommendations.
>>
>> When something is identified as "stable" or "maintained", it means the
>> development community makes a conscious effort to keep it working and
>> consider reports of bugs and issues as high priorities. Fixes
>> submitted for these issues will also be considered high priorities,
>> although they will be accepted only if they pass the general
>> acceptance criteria for new contributions (design, quality,
>> documentation, testing, etc), at the development team discretion.
>>
>>
>> Maintained projects and platforms
>>
Re: [Avocado-devel] [RFC v4]: Long Term Stability (was Avocado maintainability and integration with avocado-vt)
On 04/28/2017 11:17 AM, Jeff Nelson wrote:
> On Fri, 28 Apr 2017 15:58:11 +0200
> Amador Segundo wrote:
>
>> On Thu, Apr 27, 2017 at 5:15 PM, Cleber Rosa wrote:
>>>
>>>
>>> On 04/27/2017 04:04 AM, Amador Segundo wrote:
>>>> On Thu, Apr 27, 2017 at 1:07 AM, Cleber Rosa wrote:
>>>>> ==
>>>>> RFC: Long Term Stability
>>>>> ==
>>>>>
>>>>> This RFC contains proposals and clarifications regarding the
>>>>> maintenance and release processes of Avocado.
>>>>>
>>>>> We understand there are multiple teams currently depending on the
>>>>> stability of Avocado and we don't want their work to be disrupted by
>>>>> incompatibilities nor instabilities in new releases.
>>>>>
>>>>> This version is a minor update to previous versions of the same RFC
>>>>> (see `Changelog`_) which drove the release of Avocado 36.0 LTS. The
>>>>> Avocado team has plans for a new LTS release in the near future, so
>>>>> please consider reading and providing feedback on the proposals here.
>>>>>
>>>>> TL;DR
>>>>> =
>>>>>
>>>>> We plan to keep the current approach of sprint releases every 3-4
>>>>> weeks, but we're introducing "Long Term Stability" releases which
>>>>> should be adopted in production environments where users can't keep up
>>>>> with frequent upgrades.
>>>>>
>>>>> Changelog
>>>>> =
>>>>>
>>>>> Changes from `Version 3`_:
>>>>>
>>>>> * Converted formatting to REStructuredText
>>>>> * Replaced "me" mentions on version 1 changelog with proper name
>>>>>(Ademar Reis)
>>>>> * Renamed section "Misc Details" to `Deployment Details`_
>>>>> * Renamed "avocado-vt" to "Avocado-VT"
>>>>> * Start the timeline example with version 36.0
>>>>> * Be explicit on timeline example that a minor bug did not generate
>>>>>an immediate release
>>>>>
>>>>> Changes from `Version 2`_:
>>>>>
>>>>> * Wording changes on second paragraph ("... nor instabilities...")
>>>>> * Clarified on "Introduction" that change of behavior is introduced
>>>>>between regular releases
>>>>> * Updated distro versions for which official packages are built
>>>>> * Add more clear explanation on official packages on the various
>>>>>hardware platforms
>>>>> * Used more recent version numbers as examples, and the planned
>>>>>new LTS version too
>>>>> * Explain how users can get the LTS version when using tools such as
>>>>>pip
>>>>> * Simplified the timeline example, with examples that will possibly
>>>>>match the future versions and releases
>>>>> * Documented current status of Avocado-VT releases and packages
>>>>>
>>>>> Changes from `Version 1`_:
>>>>>
>>>>> * Changed "Support" to "Stability" and "supported" to "maintained"
>>>>>[Jeff Nelson]
>>>>> * Misc improvements and clarifications in the
>>>>>supportability/stability statements [Jeff Nelson, Ademar Reis]
>>>>> * Fixed a few typos [Jeff Nelson, Ademar Reis]
>>>>>
>>>>> Introduction
>>>>>
>>>>>
>>>>> We make new releases of Avocado every 3-4 weeks on average. In theory
>>>>> at least, we're very careful with backwards compatibility. We test
>>>>> Avocado for regressions and we try to document any issues, so
>>>>> upgrading to a new version should be (again, in theory) safe.
>>>>>
>>>>> But in practice both intended and unintended changes are introduced
>>>>> during development, and both can be frustrating for conservative
>>>>> users. We also understand it's not feasible for users to upgrade
>>>>> Avocado very frequently in a production environment.
>>>>>
>>>>> The objective of this RFC is to clarify our maintenance practices and
>>>>> introduce Long Term
Re: [Avocado-devel] [RFC v4]: Long Term Stability (was Avocado maintainability and integration with avocado-vt)
On 04/28/2017 11:17 AM, Jeff Nelson wrote: > > > Maybe I'm being pedantic, but how about some visual separation between > the sprint release and the things that follow? It looks to me as if > one could interpret everything under a numbered sprint release as > happening at the same time, when in fact we want to point out that > (a) bug discovery can happen at any time > (b) the bugfix occurs next > (c) the severity of the defect determines the timing of the release > (c.1) moderate and minor bugfixes to lts branches are held until the > next sprint release > (c.2) critical bugs are released asynchronously, without waiting for the > next sprint release. > > 53 > * sprint release 53.0 > > * moderate bug that affects users of master, 52lts and 36lts is found > * fix is added to master and backported to 52lts and 36lts > > 54 > * sprint release 54.0 **AND** 36lts release 36.5 **AND** 52lts release 52.1 > > * minor bug that affects users of master and 52lts is found > * fix is added to master and backported to 52lts > > 55 > * sprint release 55.0 **AND** 52lts release 52.2 > > * critical bug is found that affects users of 52lts *only* > * fix is made to 52lts and 52lts release 52.3 is immediately announced > > I dunno; just something to think about. > > -Jeff > This is how I managed (I hope!) to accommodate your suggestions: --- Timeline example Consider the release numbers as date markers. The bullet points beneath them are information about the release itself or events that can happen anytime between one release and the other. Assume each sprint is taking 3 weeks. 36.0 * **LTS** release (the only LTS release available at the time of writing) 37.0 .. 49.0 * sprint releases * 36.1 LTS release * 36.2 LTS release * 36.3 LTS release * 36.4 LTS release 50.0 * sprint release * start preparing a LTS release, so 51.0 will be a **beta LTS** 51.0 * sprint release * **beta LTS** release 52.0 * **LTS** release * 52lts branch is created * packages go into LTS repo * both **36.x LTS** and **52.x LTS** maintained from this point on 53.0 * sprint release * minor bug that affects 52.0 is found, fix gets added to master and 52lts branches * bug does **not** affect 36.x LTS, so a backport is **not** added to the 36lts branch 54.0 * sprint release 54.0 * LTS release 52.1 * minor bug that also affects 52.x LTS and 36.x LTS is found, fix gets added to master, 52lts and 36lts branches 55.0 * sprint release * LTS release 36.5 * LTS release 52.2 * critical bug that affects 52.2 *only* is found, fix gets added to 52lts and **52.3 LTS is immediately released** 56.0 * sprint release 57.0 * sprint release 58.0 * sprint release 59.0 * sprint release * EOL for **36.x LTS** (18 months since the release of 36.0), 36lts branch is frozen permanently. A few points are worth taking notice here: * Multiple LTS releases can co-exist before EOL * Bug discovery can happen at any time * The bugfix occurs ASAP after its discovery * The severity of the defect determines the timing of the release - moderate and minor bugfixes to lts branches are held until the next sprint release - critical bugs are released asynchronously, without waiting for the next sprint release --- Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] [RFC v5]: Long Term Stability (was Avocado maintainability and integration with avocado-vt)
- moderate and minor bugfixes to lts branches are held until the
next sprint release
- critical bugs are released asynchronously, without waiting for
the next sprint release
Avocado-VT
==
Avocado-VT is an Avocado plugin that allows "VT tests" to be run
inside Avocado. It's a third-party project maintained mostly by
Engineers from Red Hat QE with assistance from the Avocado team
and other community members.
It's a general consensus that QE teams use Avocado-VT directly
from git, usually following the master branch, which they
control.
There's no official maintenance or stability statement for
Avocado-VT. Even though the upstream community is quite
friendly and open to both contributions and bug reports,
Avocado-VT is made available without any promises for
compatibility or supportability.
When packaged and versioned, Avocado-VT rpms should be considered
just snapshots, available in packaged form as a convenience to
users outside of the Avocado-VT development community. Again,
they are made available without any promises of compatibility or
stability.
* Which Avocado version should be used by Avocado-VT?
This is up to the Avocado-VT community to decide, but the
current consensus is that to guarantee some stability in
production environments, Avocado-VT should stick to a specific
LTS release of Avocado. In other words, the Avocado team
recommends production users of Avocado-VT not to install Avocado
from its master branch or upgrade it from Sprint Releases.
Given each LTS release will be maintained for 18 months, it
should be reasonable to expect Avocado-VT to upgrade to a new
LTS release once a year or so. This process will be done with
support from the Avocado team to avoid disruptions, with proper
coordination via the avocado mailing lists.
In practice the Avocado development team will keep watching
Avocado-VT to detect and document incompatibilities, so when
the time comes to do an upgrade in production, it's expected
that it should happen smoothly.
* Will it be possible to use the latest Avocado and Avocado-VT
together?
Users are welcome to *try* this combination. The Avocado
development team itself will do it internally as a way to monitor
incompatibilities and regressions.
Whenever Avocado is released, a matching versioned snapshot of
Avocado-VT will be made. Packages containing those Avocado-VT
snapshots, for convenience only, will be made available in the
regular Avocado repository.
Changelog
=
Changes from `Version 4`_:
* Moved changelog to the bottom of the document
* Changed wording on bug handling for LTS releases ("important
issues")
* Removed ppc64 (big endian) from list of platforms
* If bugs also affect older LTS release during the transition period,
a backport will also be added to the corresponding branch
* Further work on the `Timeline example`_, adding summary of
important points and more release examples, such as the whole list
of 36.x releases and the (fictional) 36.5 and 52.3
Changes from `Version 3`_:
* Converted formatting to REStructuredText
* Replaced "me" mentions on version 1 changelog with proper name
(Ademar Reis)
* Renamed section "Misc Details" to `Deployment Details`_
* Renamed "avocado-vt" to "Avocado-VT"
* Start the timeline example with version 36.0
* Be explicit on timeline example that a minor bug did not generate
an immediate release
Changes from `Version 2`_:
* Wording changes on second paragraph ("... nor instabilities...")
* Clarified on "Introduction" that change of behavior is introduced
between regular releases
* Updated distro versions for which official packages are built
* Add more clear explanation on official packages on the various
hardware platforms
* Used more recent version numbers as examples, and the planned
new LTS version too
* Explain how users can get the LTS version when using tools such as
pip
* Simplified the timeline example, with examples that will possibly
match the future versions and releases
* Documented current status of Avocado-VT releases and packages
Changes from `Version 1`_:
* Changed "Support" to "Stability" and "supported" to "maintained"
[Jeff Nelson]
* Misc improvements and clarifications in the
supportability/stability statements [Jeff Nelson, Ademar Reis]
* Fixed a few typos [Jeff Nelson, Ademar Reis]
.. _Version 1:
https://www.redhat.com/archives/avocado-devel/2016-April/msg6.html
.. _Version 2:
https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html
.. _Version 3:
https://www.redhat.com/archives/avocado-devel/2017-April/msg00032.html
.. _Version 4:
https://www.redhat.com/archives/avocado-devel/2017-April/msg00041.html
.. _PyPI: https://pypi.python.org/pypi
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
signature.asc
Description: OpenPGP digital signature
[Avocado-devel] Next LTS (Long Term Stability) release heads up (52.0)
Hey everyone, There has been some recent discussions on the mailing list about the whole concept of "Long Term Stability"[1]. With that, users should be pretty well informed about what to expect from it. And it's our sincere hope that we managed to address the community's needs. So, it's now time for a proper announcement so that users can be prepared for the next LTS release. In short: = Version 52.0 will be the next LTS release = Now, unto the FAQ section: When is this going to be released? -- The next LTS release, 52.0, is expected to happen on June 19th, 2017. For how long will it receive stability updates? --- It will receive stability for 18 months, that is, it will reach EOL around December 18th, 2018. Will users have to switch immediately from 36.x to 52.x? No, users currently relying on the 36.x series (the current LTS version) will continue to receive stability updates for another six months, that is, until around December 18th, 2017. Be advised, though, that if you're using the official Avocado project LTS RPM package repo[2] and do a system update, you will receive, by default, the packages from the next LTS release. Consult your package manager documentation for how to pin your system to a specific package version. How ho I get my feature in 52.0? Short answer: by sending your RFEs and patches as soon as possible! Long answer: version 51.0 will be a "LTS beta", and possibly your last change of getting new features in for the 52.x series. If the core Avocado team is unable to develop your RFE until 50.0, it's suggested that you help the development process (send your patches!) for a better shot of your feature making into 52.0. References: --- [1] - https://www.redhat.com/archives/avocado-devel/2017-April/msg00050.html [2] - http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#fedora-from-avocado-s-own-repo -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 50.0: A Dog's Will
Hello everyone, This is yet another Avocado release announcement! Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/50.0/release_notes/50_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/50.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] RFC: Guidelines for categorizing tests
Introduction Avocado allows users to select tests to run based on free form "tags". These tags are given as "docstring directives", that is, special entries on a class or function docstring. As a user of an Avocado based test suite, I'd see value in not **having** to look at all the test tags before realizing that to not run tests that require "super user" permissions I should run:: $ avocado run test.py --filter-by-tags=-root Instead of:: $ avocado run test.py --filter-by-tags=-privileged Not even that, by having different tests as part of the same job, the following very odd sequence of command line options may be needed:: $ avocado run test.py test2.py --filter-by-tags=-root,-privileged So the goal here is to let users familiar with a given Avocado based test, to have fair expectations when running another Avocado based tests. This was initially going to be a documentation update, but I felt that it was not fair to make a formal proposal without without some initial brainstorming. Proposal To set the tone for my proposal, I'd like to make most things simple and easy, while allowing for "everything else" to be doable. My general impression is that higher level information about the test itself and its requirements are going to be the most commonly used tags, so they must be easily set. Some examples follow. Simple (standalone) tags Tags by functional area: * cpu - Exercises a system's CPU * net - Exercises a system's network devices or networking stack * storage - Exercises a system's local storage * fs - Exercises a system's file system Tags by architecture: * x86_64 - Requires a x86_64 architecture * ppc64 - Requries a ppc64 Tags by access privileges: * privileged - requires the test to be run with the most privileged, unrestricted privileges. For Linux systems, this usually means the root account Composed (key:value) tags - The more specific tags can be achieved by composing a predefined key with a value. For instance, to tag a test as needing a specific CPU flag: * cpu_flag:vmx Or a specific PCI device: * pci_id:8086:08b2 Or even a software package: * package:gcc Or a package group altogether: * package_group:development-tools Some examples - * ``cpu,x86_64`` - The test exercises the CPU and requires a ``x86_64`` based platform * ``net,privileged,pci_id:14e4:1657`` - The test exercises either a network device or the network stack, needs super user privileges and a "Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)" device. Looking at test tags Currently, there's no way to actually list the test tags from the command line, alongside the test name themselves. An RFE for fixing this has been filed at https://trello.com/c/NXrbKEJC . Do users have to provide all the ``--filter-by-tags`` themselves? = The test runner can certainly help here, getting system information when the job starts, and feeding them to the filtering. This is yet another reason why coming up with a good set of guidelines for tagging tests is important. In some ways, this can be seen similar to a dependency resolution mechanism for tests, only that at this point it will not resolve the requirements. It will only filter out tests that can't (or shouldn't) be loaded on the current system. Effectively, instead of many in-tests checks, and many SKIPs/CANCELs, the system information can be loaded once, and the only relevant tests will be part of the tests suite. A list of the tests that were filtered out from the job test suite can certainly be a useful part of the job results. As in every RFC, feedback is extremely welcome! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] RFC: Guidelines for categorizing tests
On 05/18/2017 12:22 AM, Narasimhan V wrote: > Hi Cleber, > > I agree the proposal. > Please find some of my comments in-line. > Hi Narasimhan, Thanks for the feedback. My responses are inline. > On 2017-05-18 03:19, Cleber Rosa wrote: >> Introduction >> >> >> Avocado allows users to select tests to run based on free form "tags". >> These tags are given as "docstring directives", that is, special >> entries on a class or function docstring. > > I like this kind of a proposal :-) > I'm not sure if it was clear enough, but this functionality already exists in Avocado: http://avocado-framework.readthedocs.io/en/50.0/WritingTests.html#categorizing-tests This proposal is about proposing a set of guidelines for using that feature. >> >> As a user of an Avocado based test suite, I'd see value in not **having** >> to look at all the test tags before realizing that to not run tests that >> require "super user" permissions I should run:: >> >> $ avocado run test.py --filter-by-tags=-root >> >> Instead of:: >> >> $ avocado run test.py --filter-by-tags=-privileged >> >> Not even that, by having different tests as part of the same job, >> the following very odd sequence of command line options may be >> needed:: >> >> $ avocado run test.py test2.py --filter-by-tags=-root,-privileged >> >> So the goal here is to let users familiar with a given Avocado based >> test, to have fair expectations when running another Avocado based >> tests. >> >> This was initially going to be a documentation update, but I felt that >> it was not fair to make a formal proposal without without some initial >> brainstorming. >> >> Proposal >> >> >> To set the tone for my proposal, I'd like to make most things simple >> and easy, while allowing for "everything else" to be doable. >> >> My general impression is that higher level information about the test >> itself and its requirements are going to be the most commonly used >> tags, so they must be easily set. Some examples follow. >> >> Simple (standalone) tags >> >> >> Tags by functional area: >> >> * cpu - Exercises a system's CPU >> * net - Exercises a system's network devices or networking stack >> * storage - Exercises a system's local storage >> * fs - Exercises a system's file system >> >> Tags by architecture: >> >> * x86_64 - Requires a x86_64 architecture >> * ppc64 - Requries a ppc64 >> >> Tags by access privileges: >> >> * privileged - requires the test to be run with the most privileged, >>unrestricted privileges. For Linux systems, this usually means the >>root account >> > > How are these tags going to be differentiated, ie, tag names that can be > common across types/areas. > Examples, if already on your mind, can help here. > Well, that's exactly the point of this proposal. If users have access to two different test repos, and both are related to hardware/software testing, they can expect that by choosing only tests with "cpu" tags, they will be exercising their Central Processing Units. Conversely, if a test repo is concerned about testing Hospital procedures (this is very hypothetical), they may choose to use "cpu" as a shorthand for "Care of Patients with Urgency". They would be going against the general Avocado guidelines, users may expect those tests to be about "Central Processing Units", and be frustrated. Does this example make sense? >> Composed (key:value) tags >> - >> >> The more specific tags can be achieved by composing a predefined key >> with a value. For instance, to tag a test as needing a specific >> CPU flag: >> >> * cpu_flag:vmx >> >> Or a specific PCI device: >> >> * pci_id:8086:08b2 >> >> Or even a software package: >> >> * package:gcc >> >> Or a package group altogether: >> >> * package_group:development-tools >> >> Some examples >> - >> >> * ``cpu,x86_64`` - The test exercises the CPU and requires a >>``x86_64`` based platform >> >> * ``net,privileged,pci_id:14e4:1657`` - The test exercises either a >>network device or the network stack, needs super user privileges >>and a "Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe >>(rev 01)" device. > > I would be intereste
Re: [Avocado-devel] RFC: Guidelines for categorizing tests
On 05/18/2017 09:42 AM, Balamuruhan S wrote: > This feature looks to be great and I acknowledge this RFC, > Hi Bala, Thanks for the feedback. My responses are inline. > we have to consider two things, > 1. with respect to avocado-vt, whether tags mentioned should filter the tests > based on > host or guest. > Right now, this wouldn't apply to Avocado-VT, only to INSTRUMENTED tests. See the documentation on the already existing tagging/filtering feature here: http://avocado-framework.readthedocs.io/en/50.0/WritingTests.html#categorizing-tests > 2. Negative test scenarios should not be filtered based on this tags. > Can you expand on this, maybe with a use case example? Sorry but I failed to understand your point. > by considering these in design for implementation helps us to achieve this > feature well. > > Regards, > Bala > Cheers! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] RFC: Guidelines for categorizing tests
On 05/18/2017 03:34 PM, Narasimhan V wrote: > On 2017-05-18 20:18, Cleber Rosa wrote: >> On 05/18/2017 12:22 AM, Narasimhan V wrote: >>> Hi Cleber, >>> >>> I agree the proposal. >>> Please find some of my comments in-line. >>> >> >> Hi Narasimhan, >> >> Thanks for the feedback. My responses are inline. >> >>> On 2017-05-18 03:19, Cleber Rosa wrote: >>>> Introduction >>>> >>>> >>>> Avocado allows users to select tests to run based on free form "tags". >>>> These tags are given as "docstring directives", that is, special >>>> entries on a class or function docstring. >>> >>> I like this kind of a proposal :-) >>> >> >> >> I'm not sure if it was clear enough, but this functionality already >> exists in Avocado: >> >> http://avocado-framework.readthedocs.io/en/50.0/WritingTests.html#categorizing-tests >> >> >> This proposal is about proposing a set of guidelines for using that >> feature. >> > > Thanks for pointing this out. > >>>> >>>> As a user of an Avocado based test suite, I'd see value in not >>>> **having** >>>> to look at all the test tags before realizing that to not run tests >>>> that >>>> require "super user" permissions I should run:: >>>> >>>> $ avocado run test.py --filter-by-tags=-root >>>> >>>> Instead of:: >>>> >>>> $ avocado run test.py --filter-by-tags=-privileged >>>> >>>> Not even that, by having different tests as part of the same job, >>>> the following very odd sequence of command line options may be >>>> needed:: >>>> >>>> $ avocado run test.py test2.py --filter-by-tags=-root,-privileged >>>> >>>> So the goal here is to let users familiar with a given Avocado based >>>> test, to have fair expectations when running another Avocado based >>>> tests. >>>> >>>> This was initially going to be a documentation update, but I felt that >>>> it was not fair to make a formal proposal without without some initial >>>> brainstorming. >>>> >>>> Proposal >>>> >>>> >>>> To set the tone for my proposal, I'd like to make most things simple >>>> and easy, while allowing for "everything else" to be doable. >>>> >>>> My general impression is that higher level information about the test >>>> itself and its requirements are going to be the most commonly used >>>> tags, so they must be easily set. Some examples follow. >>>> >>>> Simple (standalone) tags >>>> >>>> >>>> Tags by functional area: >>>> >>>> * cpu - Exercises a system's CPU >>>> * net - Exercises a system's network devices or networking stack >>>> * storage - Exercises a system's local storage >>>> * fs - Exercises a system's file system >>>> >>>> Tags by architecture: >>>> >>>> * x86_64 - Requires a x86_64 architecture >>>> * ppc64 - Requries a ppc64 >>>> >>>> Tags by access privileges: >>>> >>>> * privileged - requires the test to be run with the most privileged, >>>>unrestricted privileges. For Linux systems, this usually means the >>>>root account >>>> >>> >>> How are these tags going to be differentiated, ie, tag names that can be >>> common across types/areas. >>> Examples, if already on your mind, can help here. >>> >> >> Well, that's exactly the point of this proposal. If users have access >> to two different test repos, and both are related to hardware/software >> testing, they can expect that by choosing only tests with "cpu" tags, >> they will be exercising their Central Processing Units. >> >> Conversely, if a test repo is concerned about testing Hospital >> procedures (this is very hypothetical), they may choose to use "cpu" as >> a shorthand for "Care of Patients with Urgency". They would be going >> against the general Avocado guidelines, users may expect those tests to >> be about "Central Processing Units", and be frustrated. >> >> Does this example make sens
Re: [Avocado-devel] RFC: Guidelines for categorizing tests
On 05/18/2017 01:07 AM, Satheesh Rajendran wrote: > On Wed, 2017-05-17 at 17:49 -0400, Cleber Rosa wrote: >> Introduction >> >> >> Avocado allows users to select tests to run based on free form >> "tags". >> These tags are given as "docstring directives", that is, special >> entries on a class or function docstring. >> > This would be very helpful and becomes easy way to > create a dynamic testsuite based on needs. > +1 I'm not sure if it was clear enough, but this functionality already exists in Avocado: http://avocado-framework.readthedocs.io/en/50.0/WritingTests.html#categorizing-tests This proposal is about proposing a set of guidelines for using that feature. >> As a user of an Avocado based test suite, I'd see value in not >> **having** >> to look at all the test tags before realizing that to not run tests >> that >> require "super user" permissions I should run:: >> >> $ avocado run test.py --filter-by-tags=-root >> >> Instead of:: >> >> $ avocado run test.py --filter-by-tags=-privileged >> >> Not even that, by having different tests as part of the same job, >> the following very odd sequence of command line options may be >> needed:: >> >> $ avocado run test.py test2.py --filter-by-tags=-root,-privileged >> >> So the goal here is to let users familiar with a given Avocado based >> test, to have fair expectations when running another Avocado based >> tests. >> >> This was initially going to be a documentation update, but I felt >> that >> it was not fair to make a formal proposal without without some >> initial >> brainstorming. >> >> Proposal >> >> >> To set the tone for my proposal, I'd like to make most things simple >> and easy, while allowing for "everything else" to be doable. >> >> My general impression is that higher level information about the test >> itself and its requirements are going to be the most commonly used >> tags, so they must be easily set. Some examples follow. >> >> Simple (standalone) tags >> >> >> Tags by functional area: >> >> * cpu - Exercises a system's CPU >> * net - Exercises a system's network devices or networking stack >> * storage - Exercises a system's local storage >> * fs - Exercises a system's file system >> >> Tags by architecture: >> >> * x86_64 - Requires a x86_64 architecture >> * ppc64 - Requries a ppc64 >> >> Tags by access privileges: >> >> * privileged - requires the test to be run with the most privileged, >>unrestricted privileges. For Linux systems, this usually means >> the >>root account >> >> Composed (key:value) tags >> - >> >> The more specific tags can be achieved by composing a predefined key >> with a value. For instance, to tag a test as needing a specific >> CPU flag: >> >> * cpu_flag:vmx >> >> Or a specific PCI device: >> >> * pci_id:8086:08b2 >> >> Or even a software package: >> >> * package:gcc >> >> Or a package group altogether: >> >> * package_group:development-tools >> >> Some examples >> - >> >> * ``cpu,x86_64`` - The test exercises the CPU and requires a >>``x86_64`` based platform >> >> * ``net,privileged,pci_id:14e4:1657`` - The test exercises either a >>network device or the network stack, needs super user privileges >>and a "Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe >>(rev 01)" device. >> >> Looking at test tags >> >> >> Currently, there's no way to actually list the test tags from the >> command line, alongside the test name themselves. An RFE for fixing >> this has been filed at https://trello.com/c/NXrbKEJC . >> >> Do users have to provide all the ``--filter-by-tags`` themselves? >> = >> >> The test runner can certainly help here, getting system information >> when the job starts, and feeding them to the filtering. This is yet >> another reason why coming up with a good set of guidelines for >> tagging >> tests is important. >> >> In some ways, this can be seen similar to a dependency resolution >> mechanism for tests, only that at this point it will not resolve the >> requirements. It w
Re: [Avocado-devel] RFC: Guidelines for categorizing tests
On 05/22/2017 02:00 PM, Ademar Reis wrote: > On Wed, May 17, 2017 at 05:49:36PM -0400, Cleber Rosa wrote: >> Introduction >> >> >> Avocado allows users to select tests to run based on free form "tags". >> These tags are given as "docstring directives", that is, special >> entries on a class or function docstring. >> >> As a user of an Avocado based test suite, I'd see value in not **having** >> to look at all the test tags before realizing that to not run tests that >> require "super user" permissions I should run:: >> >> $ avocado run test.py --filter-by-tags=-root >> >> Instead of:: >> >> $ avocado run test.py --filter-by-tags=-privileged >> >> Not even that, by having different tests as part of the same job, >> the following very odd sequence of command line options may be >> needed:: >> >> $ avocado run test.py test2.py --filter-by-tags=-root,-privileged >> >> So the goal here is to let users familiar with a given Avocado based >> test, to have fair expectations when running another Avocado based >> tests. >> >> This was initially going to be a documentation update, but I felt that >> it was not fair to make a formal proposal without without some initial >> brainstorming. >> >> Proposal >> >> >> To set the tone for my proposal, I'd like to make most things simple >> and easy, while allowing for "everything else" to be doable. >> >> My general impression is that higher level information about the test >> itself and its requirements are going to be the most commonly used >> tags, so they must be easily set. Some examples follow. >> >> Simple (standalone) tags >> >> >> Tags by functional area: >> >> * cpu - Exercises a system's CPU >> * net - Exercises a system's network devices or networking stack >> * storage - Exercises a system's local storage >> * fs - Exercises a system's file system >> >> Tags by architecture: >> >> * x86_64 - Requires a x86_64 architecture >> * ppc64 - Requries a ppc64 >> >> Tags by access privileges: >> >> * privileged - requires the test to be run with the most privileged, >>unrestricted privileges. For Linux systems, this usually means the >>root account >> >> Composed (key:value) tags >> - >> >> The more specific tags can be achieved by composing a predefined key >> with a value. For instance, to tag a test as needing a specific >> CPU flag: >> >> * cpu_flag:vmx >> >> Or a specific PCI device: >> >> * pci_id:8086:08b2 >> >> Or even a software package: >> >> * package:gcc >> >> Or a package group altogether: >> >> * package_group:development-tools >> >> Some examples >> - >> >> * ``cpu,x86_64`` - The test exercises the CPU and requires a >>``x86_64`` based platform >> >> * ``net,privileged,pci_id:14e4:1657`` - The test exercises either a >>network device or the network stack, needs super user privileges >>and a "Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe >>(rev 01)" device. > > In my understanding, one of the key design aspects of tags is > that they're user-defined, optional, and totally arbitrary to the > rest of Avocado. In other words, to Avocado there's no semantics > in a tag called "ppc64", "priviledged" or "pci_id:8086:08b2". > Right, the fact that there are no semantics in tags, and that Avocado itself will *not* attempt to interpret the tags has been repeated a couple of times. I hope it's somewhat clearer now. > This should be clear in the documentation and in this RFC, > otherwise users might be tempted to start tagging tests following > some sort of "official list of tags" provided by Avocado, or > coding features that depend on some specific tag. > Users tagging tests following some sort of "official list of tags" is indeed what I'm pursuing here. And I'm pursuing it here because there has been an understanding that it'd be beneficial across test repos. BTW, it'd be an "official" *recommendation*, but not a requirement. > (or is this actually your intent?) > Partially, yes. See answer above. BTW, the task generated this RFC is called "Created a guideline for tagging tests": https://trello.com/c/ZMmFqA5j/880-created-a-guideline-for-tag
Re: [Avocado-devel] [RFC] Recursive Test Discovery
ering. Example: >> >> File `/usr/share/avocado/tests/test_base_class.py`:: >> >> from avocado import Test >> >> >> class BaseClass(Test): >> >> def test_basic(self): >> pass >> >> File `/usr/share/avocado/tests/test_first_child.py`:: >> >> from test_base_class import BaseClass >> >> >> class FirstChild(BaseClass): >> """ >> :avocado: recursive >> """ >> >> def test_first_child(self): >> pass >> >> Will result in:: >> >> $ avocado list test_first_child.py >> INSTRUMENTED test_first_child.py:FirstChild.test_first_child >> INSTRUMENTED test_first_child.py:BaseClass.test_basic >> >> While: >> >> File `/usr/share/avocado/tests/test_base_class.py`:: >> >> from avocado import Test >> >> >> class BaseClass(Test): >> """ >> :avocado: disable >> """ >> >> def test_basic(self): >> pass >> >> File `/usr/share/avocado/tests/test_first_child.py`:: >> >> from test_base_class import BaseClass >> >> >> class FirstChild(BaseClass): >> """ >> :avocado: recursive >> """ >> >> def test_first_child(self): >> pass >> >> Will result in:: >> >> $ avocado list test_first_child.py >> INSTRUMENTED test_first_child.py:FirstChild.test_first_child >> >> The alternative option is that the discovery ignores the parents >> docstrings when discovering recursively, meaning that the >> `:avocado: disable` (or any other current or future available >> docstrings) would have no effect in the recursive discovery. >> I believe my previous comments address these, albeit not specifically, as the topic I consider the most important is whether Avocado can implement the same standard Python behavior first. >> Expected Results >> >> >> The expected results of this RFC is to have a well defined behavior for >> the recursive discovery feature. >> My understanding is that we should aim towards the natural Python class behavior. >> The expected result of the feature itself is to provide users more >> flexibility when creating the Avocado tests and consequent Avocado >> command lines. >> And I do think within the same work we can improve over the "standard" behavior, such as listing only local methods as mentioned before, making Avocado more flexible than other tools. >> Additional Information >> == >> >> Avocado uses only static analysis to examine the files and this feature >> should stick to this principle in its implementation. Agreed. >> >> >> >> >> Looking forward to read your comments. >> -- >> apahim > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Make optional interfaces implementation for plugin system.
On 06/06/2017 08:48 AM, Andrei Stepanov wrote: > Hello. > > Currently Avocado Plugin System requires to implements all virtual > methods in plugins. Can we remove this restriction? RFE: Allow to > implement only necessary interfaces for plugins. Just skip their > invocation of absent methods in plugins? > Some background information may be helpful here. As you most certainly know, Python[1] doesn't have any builtin solution for plugins/extensions. This is both good and bad. Initially, Avocado chose to write its own plugin/extension support code. Besides the extra code we had to maintain, it also had many shortcomings. So we went shopping for a better solution, and found/selected stevedore[2]. One of the stevedore suggestions, is to use Abstract Base Classes, which "provides a convenient way to document the API" and "keeps you honest"[3]. Since we benefited from its code, which is directly related to its accumulated experience, we decided to follow its advice. Additionally, this is clearly a trade off situation. It'd either require the complete interface to be present in the implementation (even if it does nothing) or to check the presence of each implementation method. We chose the former. IMO, it's hard to make a point when the proposal is to move the problem/complexity elsewhere. If we can simplify this, without adding other code elsewhere, I'm all in. > 1. This will allow as easily add new methods to existing Core Interfaces: > > http://avocado-framework.readthedocs.io/en/latest/api/core/avocado.core.html#module-avocado.core.plugin_interfaces > > Otherwise, any new interface extension to Core will force us to > rewrite __ALL__ plugins, which makes Avocado quite inflexible for > upgrading and for external plugins. > The plugin interfaces are currently *core*. It means that, in theory, there's no stability support in those interfaces and that users indeed have to pay attention to Avocado upgrades. In practice, there has been very little changes to those interfaces, and, if we feel the demand, we can consider making those interfaces stable in the future. Talking about interfaces and stability, that's exactly why we chose to define and require an interface. IMO, when you require a set of methods implemented, you minimize misbehavior from plugins on newer different Avocado versions. > 2. Allow for plugin-developers implement methods they require. > Yes, the current solution has indeed the requirement that all methods must exist, even if they do nothing. The reasons for those requirements have already been given. I honestly don't see the reason for a change of behavior here. But if other users/developers out there feel this is something that should be changed, this is the perfect time for them to speak up. > Thanks. > Thank you! [1] - Meaning the language, the CPython implementation and its standard library [2] - http://git.openstack.org/cgit/openstack/stevedore/ [3] - https://docs.openstack.org/developer/stevedore/tutorial/creating_plugins.html#creating-plugins -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado Sprint #51 Release Meeting
Dear Avocado users and developers, I'd like to invite you all to a sprint release and planning meeting. It is going to take place at date -d "Mon Jun 12 10:00:08 EDT 2017" using the Hangouts on Air. The link to join the meeting will be shared 10min in advance as a reply for this e-mail and in our irc channel. You can also watch the meeting in the link below: https://www.youtube.com/watch?v=NWfOvo2gWhE And ask questions via IRC irc://irc.oftc.net/#avocado The meeting will be split in two parts (roughly 30 min each): * Part 1: Sprint Review * Review of the changes introduced during this sprint * Short demonstrations of some of the new features * Live sprint status: release readiness, last minute blockers, etc. * Part 2: Sprint Planning * New sprint planning boards will be created and its tasks prioritized. * Quick review of the expectations for the next sprint (high-level vision, goals, highlights) The meeting is going to be recorded and shared on YouTube afterwards. It is worth mentioning that most of the tasks during the meeting are going to take place on our Trello board: https://trello.com/b/WbqPNl2S/avocado See you there. Best Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Pre-Release 51.0 Test Plan Results (Minor Failures)
Result is: minor failures, addressed in https://github.com/avocado-framework/avocado/pull/2063 Test Plan: Release Test Plan Run by 'cleber' at 2017-06-11T18:55:46.782231 FAIL: 'Avocado source is sound': Style problem: avocado/core/loader.py:662:20: W503 line break before binary operator FAIL: 'Avocado source does not contain spelling errors': Spelling failures addressed in PR FAIL: 'Avocado RPM build': Docs failed to build during RPM build, addressed in PR FAIL: 'Avocado RPM lint': W: incoherent-version-in-changelog 50.0-1 ['50.0-1.20170612gitac2e3c8.fc25', '50.0-1.20170612gitac2e3c8'] (this can be ignored) PASS: 'Avocado RPM install': PASS: 'Avocado Test Run on RPM based installation': PASS: 'Avocado Test Run on Virtual Machine': PASS: 'Avocado Test Run on Remote Machine': PASS: 'Avocado Remote Machine HTML report': PASS: 'Avocado Server Source Checkout and Unittests': PASS: 'Avocado Server Run': PASS: 'Avocado Server Functional Test': PASS: 'Avocado Virt and VT Source Checkout': PASS: 'Avocado Virt Bootstrap': PASS: 'Avocado Virt Boot Test Run and HTML report': PASS: 'Avocado Virt - Assignment of values from the cmdline': PASS: 'Avocado Virt - Migration test': PASS: 'Avocado VT': PASS: 'Avocado HTML report sysinfo': PASS: 'Avocado HTML report links': PASS: 'Paginator': avocado: ac2e3c83628ab3331585d58d72240f37b3661578 avocado-server: 8cec494edcf160a8bc9f61e174ff7bcedb1e6800 avocado-virt: 43c996d5060ebf3b5261faadb11b9bb0a127b8e1 avocado-virt-tests: cc185f45cdadef0898f6a382ae81179cdfa1e5df avocado-vt: 04e609ebc01a10ae97ff89595aa1e27535f611a8 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Avocado Sprint #51 Release Meeting
On 06/09/2017 12:38 PM, Cleber Rosa wrote: > Dear Avocado users and developers, > > I'd like to invite you all to a sprint release and planning meeting. > It is going to take place at date -d "Mon Jun 12 10:00:08 EDT 2017" > using the Hangouts on Air. > The link to join the meeting will be shared 10min in advance as a > reply for this e-mail and in our irc channel. You can also watch the > meeting in the link below: > > https://www.youtube.com/watch?v=NWfOvo2gWhE > If you'd rather join the meeting (instead of just watching it), join via this URL: https://hangouts.google.com/hangouts/_/7ledhp7g6fhypay53jkq2qludie > And ask questions via IRC irc://irc.oftc.net/#avocado > > The meeting will be split in two parts (roughly 30 min each): > > * Part 1: Sprint Review > * Review of the changes introduced during this sprint > * Short demonstrations of some of the new features > * Live sprint status: release readiness, last minute blockers, etc. > > * Part 2: Sprint Planning > * New sprint planning boards will be created and its tasks prioritized. > * Quick review of the expectations for the next sprint (high-level > vision, goals, highlights) > > The meeting is going to be recorded and shared on YouTube afterwards. > It is worth mentioning that most of the tasks during the meeting are > going to take place on our Trello board: > > https://trello.com/b/WbqPNl2S/avocado > > See you there. > > Best Regards, > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 51.0: The White Mountains
Hello everyone, This is yet another Avocado release announcement! Release 51.0 is out, with a large number of improvements! Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/51_0.html Upcoming LTS This is the very last release before another LTS (Long Term Stability) release. Yes, version 52.0, to be released in 2-3 weeks, will supersede 36.x as the latest LTS release. Read more about what that means here: http://avocado-framework.readthedocs.io/en/latest/rfcs/LongTermStability.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/51.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado Sprint #52 Release Meeting (52.0 LTS Release)
Dear Avocado users and developers, I'd like to invite you all to a sprint release and planning meeting. It is going to take place at date -d "Mon Jun 26 10:00:08 EDT 2017" using the Hangouts on Air. The link to join the meeting will be shared 10min in advance as a reply for this e-mail and in our irc channel. You can also watch the meeting in the link below: https://www.youtube.com/watch?v=nTeyu_XgFwM And ask questions via IRC irc://irc.oftc.net/#avocado The meeting will be split in two parts (roughly 30 min each): * Part 1: Sprint Review * Review of the changes introduced during this sprint * Short demonstrations of some of the new features * Live sprint status: release readiness, last minute blockers, etc. * Part 2: Sprint Planning * New sprint planning boards will be created and its tasks prioritized. * Quick review of the expectations for the next sprint (high-level vision, goals, highlights) The meeting is going to be recorded and shared on YouTube afterwards. It is worth mentioning that most of the tasks during the meeting are going to take place on our Trello board: https://trello.com/b/WbqPNl2S/avocado See you there. Best Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Avocado Sprint #52 Release Meeting (52.0 LTS Release)
On 06/23/2017 09:49 PM, Cleber Rosa wrote: > Dear Avocado users and developers, > > I'd like to invite you all to a sprint release and planning meeting. > It is going to take place at date -d "Mon Jun 26 10:00:08 EDT 2017" > using the Hangouts on Air. > > The link to join the meeting will be shared 10min in advance as a > reply for this e-mail and in our irc channel. You can also watch the > meeting in the link below: > > https://www.youtube.com/watch?v=nTeyu_XgFwM For those wishing to attend the meeting, here's the Hangouts URL: https://hangouts.google.com/hangouts/_/6w3o5nup6bc6nkgmrqqrb2ef3ye > > And ask questions via IRC irc://irc.oftc.net/#avocado > > The meeting will be split in two parts (roughly 30 min each): > > * Part 1: Sprint Review > * Review of the changes introduced during this sprint > * Short demonstrations of some of the new features > * Live sprint status: release readiness, last minute blockers, etc. > > * Part 2: Sprint Planning > * New sprint planning boards will be created and its tasks prioritized. > * Quick review of the expectations for the next sprint (high-level > vision, goals, highlights) > > The meeting is going to be recorded and shared on YouTube afterwards. > It is worth mentioning that most of the tasks during the meeting are > going to take place on our Trello board: > > https://trello.com/b/WbqPNl2S/avocado > > See you there. > > Best Regards, > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 52.0: Pat & Mat
Hello everyone, This is yet another Avocado release announcement! Release 52.0 is out, and it's a pretty special one. Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/52_0.html About LTS (Long Term Stability) === This release is our second LTS (Long Term Stability) release. The previous LTS release, 36.x, will be supported for an additional 6 months. Read more about what LTS means here: http://avocado-framework.readthedocs.io/en/latest/rfcs/LongTermStability.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/52.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 24 and 25. Please note that, being an LTS release, it's necessary to enable to "avocado-lts" repo if you're using the RPM package repositories. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Parameter System Overhaul
osed by local
parameter availability do *not* outweigh the problems that an external
approach can bring. In the future, if advanced use cases require an
external approach to parameters availability, this can be reevaluated.
Namespaces (AKA how/if should we merge parameters)
==
Currently, the parameter fetching interface already contains at its
core the concept of paths[1]. In theory, this is sufficient to prevent
clashes of keys with the same names, but intended to configure different
aspects of a test.
Now, with the proposed implementation of multiple providers to the
parameter system, the question of how they will be combined comes up.
One way is for each implementation to claim, based on some unique
attribute such as its own name, a part of a tree path. For instance,
for two implementations:
1) variants
2) plain_ini
Users could access parameters explicitly defined on those by referring
to paths such as:
self.params.get('speed', path='/plain_ini', default=60)
or
self.params.get('speed', path='/variants/path/inside/varianter',
default=60)
This clearly solves the clashes, but binds the implementation to the
tests, which should be avoided at all costs.
One other approach would be to merge everything into the tree root
node. By doing this, one parameter provider could effectively
override the values in another parameter provider, given that both
used the same paths for a number of parameters.
Yet another approach would be to *not* use paths, and resort to
completely separate namespaces. A parameter namespace would be an
additional level of isolation, which can quickly become exceedingly
complex.
As can be seen from the section name, I'm not proposing one solution
at this point, but hoping that a discussion on the topic would help
achieve the best possible design.
[1] -
http://avocado-framework.readthedocs.io/en/52.0/api/core/avocado.core.html#avocado.core.varianter.AvocadoParams.get
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
signature.asc
Description: OpenPGP digital signature
Re: [Avocado-devel] Cross compiling avocado and installing on another machine
On 08/01/2017 01:47 AM, Yesuraj Abraham wrote: > Hi, > > Could you help in cross compiling the avocado framework and install on > another machine. I have installed ARM cross compiling tool chain at host > machine. > > I would like to get all plugins too cross compiled and installed > > Hi Yesuraj, Avocado should be usable on all architectures with a (c)Python 2.7 distribution. What I mean is that for Avocado itself, no cross compilation is necessary. Does your OS/distro provide such a Python version? Now, some plugins have some extra requirements, such as PyYAML, which may need to be cross compiled if not available on your OS/distro. Can you share more details of your environment and the exact plugins you're trying to install? Regards, - Cleber. > > > > Regards, > > Yesuraj > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Maintainer nomination for tp-libvirt and avocado-vt
On 08/04/2017 08:05 AM, Amador Pahim wrote: > On Fri, Aug 4, 2017 at 1:27 PM, Guannan Sun wrote: >> Hi, >> >> As people change in team, new maintainers required for step up on >> maintaining avocado-vt/tp-libvirt projects. As tp-libvirt is still growing, >> you could find data from past 8 months: >> >> https://github.com/autotest/tp-libvirt/graphs/contributors?from=2016-12-31&to=2017-08-04&type=c >> >> you will find Chunfu have become the top committer on the project and he is >> really active. >> >> Kyla have been on both projects for a long time, and she spend a lot of time >> coordinate the developing work internally. >> Both kyla and chwen are willing to step up on maintainers work and commit to >> the projects. >> So at here I'm nominating them maintainers as following: >> >> tp-libvirt >> --- >> Chunfu Wen >> Kyla Zhang >> >> Avocado-vt >> --- >> Kyla Zhang >> Sure thing. I just added to autotest maintainers (for tp-libvirt): Kyla Zhang (kylazhang) Chunfu Wen (chunfuwen) And to Avocado-vt maintainers: Kyla Zhang (kylazhang) Regards! - Cleber. >> Please help with the nomination and any problem please let us know. > > Yes, well deserved. > +1 > >> >> Thanks! >> Guannan Wayne Sun >> Libvirt QE -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 53.0: Rear Window
Hello everyone, This is yet another Avocado release announcement: 53.0 is now available! Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/53_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/53.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 26 and 25. Starting with this release, there will be no more Fedora 24 packages. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Parameter System Overhaul
On 08/07/2017 05:49 PM, Ademar Reis wrote: > On Tue, Aug 01, 2017 at 03:37:34PM -0400, Cleber Rosa wrote: >> Even though Avocado has had a parameter passing system for >> instrumented tests almost from day one, it has been intertwined with >> the varianter (then multiplexer) and this is fundamentally wrong. The >> most obvious example of this broken design is the `mux-inject` command >> line option:: >> >> --mux-inject [MUX_INJECT [MUX_INJECT ...]] >> Inject [path:]key:node values into the final >> multiplex >> tree. >> >> This is broken design not because such a varianter implementations can >> be tweaked over the command line, that's fine. It's broken because it >> is the recommended way of passing parameters on the command line. >> >> The varianter (or any other subsystem) should be able to act as a >> parameter provider, but can not dictate that parameters must first be >> nodes/key/values of its own internal structure. > > Correct. It's broken because it violates several layers. There would > be nothing wrong with something like "--param [prefix:]", > for example (more below). > >> >> The proposed design >> === >> >> A diagram has been used on a few different occasions, to describe how >> the parameters and variants generation mechanism should be connected >> to a test and to the overall Avocado architecture. Here it is, in its >> original form:: >> >>+--+ >>| Test | >>+--+ >> | >> | >>+-v-+++ >>| Parameters System |--->| Variants Generation Plugin API | >>+---+++ >> ^^ >> || >>+--+ | >>| +--+ +-+ | | >>| | avocado-virt | | other providers | | | >>| +--+ +-+ | | >>+--+ | >> | >> ++-+ >> | | >> | | >> | | >>++ +-+ >>| Multiplexer Plugin | | Other variant plugin(s) | >>++ +-+ >> | >> | >>+-v---+ >>| ++ +--+ | >>| | --mux-yaml | | --mux-inject | | >>| ++ +--+ | >>+-+ >> >> >> Given that the "Parameter System" is the entry point into the parameters >> providers, it should provide two different interfaces: >> >> 1) An interface for its users, that is, developers writing >> `avocado.Test` based tests >> >> 2) An interface for developers of additional providers, such as the >> "avocado-virt" and "other providers" box on the diagram. >> >> The current state of the the first interface is the ``self.params`` >> attribute. Hopefully, it will be possible to keep its current interface, >> so that tests won't need any kind of compatibility adjustments. > > Right. The way I envision the parameters system includes a > resolution mechanism, the "path" currently used in params.get(). > This adds extra specificity to the user who requests a parameter. > > But these parameters can be provided by any entity. In the diagram > above, they're part of the "Parameter System" box. Examples of > "other providers" could be support for a configuration file or a > "--param=[prefix:]" command line option. > OK, we're in sync. > The Test API to request parameters should be shared. To a test it > doesn't matter where the parameter is coming from. They're always > accessed through an API like: > > params.get(, [prefix], [fallback value]) > As a reference to the current status, this is what this looks like: get(key, path=None, default=None) http://avocado-framework.readthedocs.io/en/53.0/api/core/avocad
Re: [Avocado-devel] Parameter System Overhaul
On 08/08/2017 07:01 AM, Lukáš Doktor wrote: > Hello guys, > > I'm sorry for such a late response, I totally forgot about this email (thanks > to Ademar, your response reminded it to me). > > Dne 7.8.2017 v 23:49 Ademar Reis napsal(a): >> On Tue, Aug 01, 2017 at 03:37:34PM -0400, Cleber Rosa wrote: >>> Even though Avocado has had a parameter passing system for >>> instrumented tests almost from day one, it has been intertwined with >>> the varianter (then multiplexer) and this is fundamentally wrong. The >>> most obvious example of this broken design is the `mux-inject` command >>> line option:: >>> >>> --mux-inject [MUX_INJECT [MUX_INJECT ...]] >>> Inject [path:]key:node values into the final >>> multiplex >>> tree. >>> >>> This is broken design not because such a varianter implementations can >>> be tweaked over the command line, that's fine. It's broken because it >>> is the recommended way of passing parameters on the command line. >>> >>> The varianter (or any other subsystem) should be able to act as a >>> parameter provider, but can not dictate that parameters must first be >>> nodes/key/values of its own internal structure. >> >> Correct. It's broken because it violates several layers. There would >> be nothing wrong with something like "--param [prefix:]", >> for example (more below). >> > Well I wouldn't call it broken. The implementation is fine we only lack other > providers which would allow to inject just params so people are abusing > `mux-inject` for that. > >>> >>> The proposed design >>> === >>> >>> A diagram has been used on a few different occasions, to describe how >>> the parameters and variants generation mechanism should be connected >>> to a test and to the overall Avocado architecture. Here it is, in its >>> original form:: >>> >>>+--+ >>>| Test | >>>+--+ >>> | >>> | >>>+-v-+++ >>>| Parameters System |--->| Variants Generation Plugin API | >>>+---+++ >>> ^^ >>> || >>>+--+ | >>>| +--+ +-+ | | >>>| | avocado-virt | | other providers | | | >>>| +--+ +-+ | | >>>+--+ | >>> | >>> ++-+ >>> | | >>> | | >>> | | >>>++ +-+ >>>| Multiplexer Plugin | | Other variant plugin(s) | >>>++ +-+ >>> | >>> | >>>+-v---+ >>>| ++ +--+ | >>>| | --mux-yaml | | --mux-inject | | >>>| ++ +--+ | >>>+-+ >>> >>> >>> Given that the "Parameter System" is the entry point into the parameters >>> providers, it should provide two different interfaces: >>> >>> 1) An interface for its users, that is, developers writing >>> `avocado.Test` based tests >>> >>> 2) An interface for developers of additional providers, such as the >>> "avocado-virt" and "other providers" box on the diagram. >>> >>> The current state of the the first interface is the ``self.params`` >>> attribute. Hopefully, it will be possible to keep its current interface, >>> so that tests won't need any kind of compatibility adjustments. >> >> Right. The way I envision the parameters system includes a >> resolution mechanism, the "path" currently used in params.get(). >> This adds extra specificity to the user who requests a parameter. >> >&
Re: [Avocado-devel] Parameter System Overhaul
On 08/16/2017 12:58 PM, Ademar Reis wrote: > On Wed, Aug 16, 2017 at 12:01:08PM -0400, Cleber Rosa wrote: >> >> >> On 08/07/2017 05:49 PM, Ademar Reis wrote: >>> On Tue, Aug 01, 2017 at 03:37:34PM -0400, Cleber Rosa wrote: >>>> Even though Avocado has had a parameter passing system for >>>> instrumented tests almost from day one, it has been intertwined with >>>> the varianter (then multiplexer) and this is fundamentally wrong. The >>>> most obvious example of this broken design is the `mux-inject` command >>>> line option:: >>>> >>>> --mux-inject [MUX_INJECT [MUX_INJECT ...]] >>>> Inject [path:]key:node values into the final >>>> multiplex >>>> tree. >>>> >>>> This is broken design not because such a varianter implementations can >>>> be tweaked over the command line, that's fine. It's broken because it >>>> is the recommended way of passing parameters on the command line. >>>> >>>> The varianter (or any other subsystem) should be able to act as a >>>> parameter provider, but can not dictate that parameters must first be >>>> nodes/key/values of its own internal structure. >>> >>> Correct. It's broken because it violates several layers. There would >>> be nothing wrong with something like "--param [prefix:]", >>> for example (more below). >>> >>>> >>>> The proposed design >>>> === >>>> >>>> A diagram has been used on a few different occasions, to describe how >>>> the parameters and variants generation mechanism should be connected >>>> to a test and to the overall Avocado architecture. Here it is, in its >>>> original form:: >>>> >>>>+--+ >>>>| Test | >>>>+--+ >>>> | >>>> | >>>>+-v-+++ >>>>| Parameters System |--->| Variants Generation Plugin API | >>>>+---+++ >>>> ^^ >>>> || >>>>+--+ | >>>>| +--+ +-+ | | >>>>| | avocado-virt | | other providers | | | >>>>| +--+ +-+ | | >>>>+--+ | >>>> | >>>> ++-+ >>>> | | >>>> | | >>>> | | >>>>++ +-+ >>>>| Multiplexer Plugin | | Other variant plugin(s) | >>>>++ +-+ >>>> | >>>> | >>>>+-v---+ >>>>| ++ +--+ | >>>>| | --mux-yaml | | --mux-inject | | >>>>| ++ +--+ | >>>>+-+ >>>> >>>> >>>> Given that the "Parameter System" is the entry point into the parameters >>>> providers, it should provide two different interfaces: >>>> >>>> 1) An interface for its users, that is, developers writing >>>> `avocado.Test` based tests >>>> >>>> 2) An interface for developers of additional providers, such as the >>>> "avocado-virt" and "other providers" box on the diagram. >>>> >>>> The current state of the the first interface is the ``self.params`` >>>> attribute. Hopefully, it will be possible to keep its current interface, >>>> so that tests won't need any kind of compatibility adjustments. >>> >>> Right. The way I envision the parameters system includes a >>> resolution mechanism, the "path" currently used in params.get(). >>> This adds extra specificity to the us
Re: [Avocado-devel] Parameter System Overhaul
On 08/08/2017 09:21 AM, Ademar Reis wrote: > On Tue, Aug 08, 2017 at 01:01:26PM +0200, Lukáš Doktor wrote: >> Hello guys, >> >> I'm sorry for such a late response, I totally forgot about this email >> (thanks to Ademar, your response reminded it to me). >> >> Dne 7.8.2017 v 23:49 Ademar Reis napsal(a): >>> On Tue, Aug 01, 2017 at 03:37:34PM -0400, Cleber Rosa wrote: >>>> Even though Avocado has had a parameter passing system for >>>> instrumented tests almost from day one, it has been intertwined with >>>> the varianter (then multiplexer) and this is fundamentally wrong. The >>>> most obvious example of this broken design is the `mux-inject` command >>>> line option:: >>>> >>>> --mux-inject [MUX_INJECT [MUX_INJECT ...]] >>>> Inject [path:]key:node values into the final >>>> multiplex >>>> tree. >>>> >>>> This is broken design not because such a varianter implementations can >>>> be tweaked over the command line, that's fine. It's broken because it >>>> is the recommended way of passing parameters on the command line. >>>> >>>> The varianter (or any other subsystem) should be able to act as a >>>> parameter provider, but can not dictate that parameters must first be >>>> nodes/key/values of its own internal structure. >>> >>> Correct. It's broken because it violates several layers. There would >>> be nothing wrong with something like "--param [prefix:]", >>> for example (more below). >>> >> Well I wouldn't call it broken. The implementation is fine we only lack >> other providers which would allow to inject just params so people are >> abusing `mux-inject` for that. >> >>>> >>>> The proposed design >>>> === >>>> >>>> A diagram has been used on a few different occasions, to describe how >>>> the parameters and variants generation mechanism should be connected >>>> to a test and to the overall Avocado architecture. Here it is, in its >>>> original form:: >>>> >>>>+--+ >>>>| Test | >>>>+--+ >>>> | >>>> | >>>>+-v-+++ >>>>| Parameters System |--->| Variants Generation Plugin API | >>>>+---+++ >>>> ^^ >>>> || >>>>+--+ | >>>>| +--+ +-+ | | >>>>| | avocado-virt | | other providers | | | >>>>| +--+ +-+ | | >>>>+--+ | >>>> | >>>> ++-+ >>>> | | >>>> | | >>>> | | >>>>++ +-+ >>>>| Multiplexer Plugin | | Other variant plugin(s) | >>>>++ +-+ >>>> | >>>> | >>>>+-v---+ >>>>| ++ +--+ | >>>>| | --mux-yaml | | --mux-inject | | >>>>| ++ +--+ | >>>>+-+ >>>> >>>> >>>> Given that the "Parameter System" is the entry point into the parameters >>>> providers, it should provide two different interfaces: >>>> >>>> 1) An interface for its users, that is, developers writing >>>> `avocado.Test` based tests >>>> >>>> 2) An interface for developers of additional providers, such as the >>>> "avocado-virt" and "other providers" box on the diagram. >>>> >>>> The current state of the the first interface is the ``self.params`` >>>>
Re: [Avocado-devel] Parameter System Overhaul
On 08/16/2017 04:34 PM, Ademar Reis wrote: > On Wed, Aug 16, 2017 at 02:28:47PM -0400, Cleber Rosa wrote: >> >> It's not covered by the fallback mechanism. What I mean is that Avocado >> should probably have, as one of the test parameter system providers >> which are enabled by default, one that looks for parameters in the test >> data dir. Something like: >> >> 1. test => examples/tests/sleeptest.py >> 2. test's datadir (current concept) => examples/tests/sleeptest.py.data >> 3. test's file-based param => examples/tests/sleeptest.py.data/params >> >> The code handling line 3, would be one of the test parameter system >> providers. Just one that is enabled by default. > > OK, this makes sense to me and is aligned with what I propose, we're > in sync. > OK. >> >>>> >>>>> Users who don't want any specificity and/or have a small test base >>>>> with a low chance of clashes could simply ignore the prefix both >>>>> when creating parameters and when making calls to params.get(). >>>>> >>>> >>>> Yes. >>>> >>>>>> >>>>>> The second item will probably mean the definition of a new class to >>>>>> the ``avocado.core.plugin_interfaces`` module, together with a new >>>>>> dispatcher(-like) implementation in ``avocado.core.dispatcher``. >>>>>> >>>>>> Parameters availability: local .vs. external >>>>>> >>>>>> >>>>>> Right now, all parameters are given to the test at instantiation time. >>>>>> Let's say that in this scenario, all parameters are *local* to the >>>>>> test. Local parameters have the benefit that the test is self >>>>>> contained and doesn't need any external communication. >>>>>> >>>>>> In theory, this is also a limitation, because all parameters must be >>>>>> available before the test is started. Even if other parameter system >>>>>> implementations are possible, with a local approach, there would be a >>>>>> number of limitations. For long running tests, that may depend on >>>>>> parameters generated during the test, this would be a blocker. Also, >>>>>> if a huge number of parameters would be available (think of a huge >>>>>> cloud or database of parameters) they would all have to be copied to >>>>>> the test at instantiation time. Finally, it also means that the >>>>>> source of parameters would need to be able to iterate over all the >>>>>> available parameters, so that they can be copied, which can be a >>>>>> further limitation for cascading implementations. >>>>>> >>>>>> An external approach to parameters, would be one that the test holds a >>>>>> handle to a broker of parameter providers. The parameter resolution >>>>>> would be done at run time. This avoids the copying of parameters, but >>>>>> requires runtime communication with the parameter providers. This can >>>>>> make the test execution much more fragile and dependent on the external >>>>>> communication. Even by minimizing the number of communication >>>>>> endpoints by communicating with the test runner only, it can still add >>>>>> significant overhead, latency and point of failures to the test >>>>>> execution. >>>>>> >>>>>> I believe that, at this time, the limitations imposed by local >>>>>> parameter availability do *not* outweigh the problems that an external >>>>>> approach can bring. In the future, if advanced use cases require an >>>>>> external approach to parameters availability, this can be reevaluated. >>>>> >>>>> If I understand your point correctly, this is an implementation >>>>> detail. It depends on what the "contract" is between the test runner >>>>> (parameter provider) and the test being run (the user of >>>>> params.get()). >>>>> >>>> >>>> It is an implementation detail, but one that will ultimately reflect on >>>> what is available to test writers. >>>> >>>>> For example, should users assume parameters are dynamic and can >>>>> change during th
Re: [Avocado-devel] Parameter System Overhaul
On 08/21/2017 08:18 AM, Lukáš Doktor wrote: > ... >>> So let me change my question: >>> >>> I've been assuming params.get() is the only params-related API we >>> *want* (or plan) to expose to test writers. Is that correct? >>> >> >> That's correct. >> >> And that's why I kept bringing back the discussion about params.iteritems(). >> > > Actually we were asked for that function as: > > 1. some users were not happy about the clash detection and they iterate > params to find the correct parameter. > 2. we need it in `dry-run` to list all params. > But that can happen outside of the API available to test writers. Based on all we have discussed so far, it's the test runner task to provide raw, already merged and processed, parameters to the test. That means it can log both things. - Cleber. > Lukáš > >>> Thanks. >>>- Ademar >>> >> > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Request to backport PR 1376 to 36lts
On 09/11/2017 10:21 AM, Lucas Meneghel Rodrigues wrote: > > > On Mon, Sep 11, 2017 at 10:23 AM Lukáš Doktor <mailto:ldok...@redhat.com>> wrote: > > Hello Guannan, > > theoretically we should not accept such backport, because it's a new > feature, not a bugfix. Anyway it touches `avocado.utils` and it's > not changing existing callbacks. What do you think, guys? In my view > the `avocado/core` is the core and I wouldn't like such changes > there, but I'd be fine with some exceptions when it goes to > `avocado/utils`. > > > I believe we should stick to the established policy. Guannan should be > able to put the needed backported functions into a testing specific > library, until he can move away from 36lts. > > Even before looking at the code that was requested to be backported, I agreed with this. I mean, if the requested backport was such that did not quite fit the bug description, but was one that was hard to work around without a backport, I'd be tempted to accept an exception to the policy. But this is not the case. IIUC, this is about avocado.utils.cpu.cpu_online_list(), a self contained, 5-liner function, that can easily be carried around with the test code. Guannan, Sorry, but we'll have to NACK this request of yours. Thanks! - Cleber. > Lukáš > > Dne 8.9.2017 v 08:41 Guannan Sun napsal(a): > > Hi, > > > > As RHEL6 still need use 36lts, and cases updated with using the > function in > > PR 1376: > > > > https://github.com/avocado-framework/avocado/pull/1376 > > > > could you help backport the commits to 36lts? > > > > Thanks! > > Guannan > > > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado Sprint #54 Release Meeting
Dear Avocado users and developers, I'd like to invite you all to a sprint release and planning meeting. It is going to take place at date -d "Tue Sep 19 10:00:00 EDT 2017" using the Hangouts on Air. The link to join the meeting will be shared 10min in advance as a reply for this e-mail and in our irc channel. You can also watch the meeting in the link below: https://www.youtube.com/watch?v=E4HpNZjBCYA And ask questions via IRC irc://irc.oftc.net/#avocado The meeting will be split in two parts (roughly 30 min each): * Part 1: Sprint Review * Review of the changes introduced during this sprint * Short demonstrations of some of the new features * Live sprint status: release readiness, last minute blockers, etc. * Part 2: Sprint Planning * New sprint planning boards will be created and its tasks prioritized. * Quick review of the expectations for the next sprint (high-level vision, goals, highlights) The meeting is going to be recorded and shared on YouTube afterwards. It is worth mentioning that most of the tasks during the meeting are going to take place on our Trello board: https://trello.com/b/WbqPNl2S/avocado See you there. Best Regards, -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Pre-release 54.0 testplan results
Result is: a couple of failures, already addressed in PRs. --- Test Plan: Release Test Plan Run by 'cleber' at 2017-09-19T06:23:31.988066 PASS: 'Avocado source is sound': FAIL: 'Avocado source does not contain spelling errors': https://github.com/avocado-framework/avocado/pull/2213 FAIL: 'Avocado RPM build': https://github.com/avocado-framework/avocado/pull/2214 PASS: 'Avocado RPM lint': python-avocado.noarch: W: incoherent-version-in-changelog 53.0-1 ['53.0-1.20170919git0c26dffb.fc26', '53.0-1.20170919git0c26dffb'] PASS: 'Avocado RPM install': PASS: 'Avocado Test Run on RPM based installation': PASS: 'Avocado Test Run on Virtual Machine': PASS: 'Avocado Test Run on Remote Machine': PASS: 'Avocado Remote Machine HTML report': PASS: 'Avocado Server Source Checkout and Unittests': PASS: 'Avocado Server Run': PASS: 'Avocado Server Functional Test': PASS: 'Avocado Virt and VT Source Checkout': PASS: 'Avocado Virt Bootstrap': PASS: 'Avocado Virt Boot Test Run and HTML report': PASS: 'Avocado Virt - Assignment of values from the cmdline': PASS: 'Avocado Virt - Migration test': PASS: 'Avocado VT': PASS: 'Avocado HTML report sysinfo': PASS: 'Avocado HTML report links': PASS: 'Paginator': avocado: 0c0d35856d29420165f83217bd40cbc2df3aa13e avocado-server: 8cec494edcf160a8bc9f61e174ff7bcedb1e6800 avocado-virt: 060352ba40b03892f27b5d7a84c468614888968e avocado-virt-tests: cc185f45cdadef0898f6a382ae81179cdfa1e5df avocado-vt: c8b227359bb3b52ddeebf9c28acf8b15277d91e2 -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Avocado Sprint #54 Release Meeting
On 09/15/2017 07:27 AM, Cleber Rosa wrote: > Dear Avocado users and developers, > > I'd like to invite you all to a sprint release and planning meeting. > It is going to take place at date -d "Tue Sep 19 10:00:00 EDT 2017" > using the Hangouts on Air. > > The link to join the meeting will be shared 10min in advance as a > reply for this e-mail and in our irc channel. You can also watch the > meeting in the link below: > > https://www.youtube.com/watch?v=E4HpNZjBCYA To join live, please use the following URL: https://hangouts.google.com/hangouts/_/mxv4se3psra6betiv7p3jy6aqae > > And ask questions via IRC irc://irc.oftc.net/#avocado > > The meeting will be split in two parts (roughly 30 min each): > > * Part 1: Sprint Review > * Review of the changes introduced during this sprint > * Short demonstrations of some of the new features > * Live sprint status: release readiness, last minute blockers, etc. > > * Part 2: Sprint Planning > * New sprint planning boards will be created and its tasks prioritized. > * Quick review of the expectations for the next sprint (high-level > vision, goals, highlights) > > The meeting is going to be recorded and shared on YouTube afterwards. > It is worth mentioning that most of the tasks during the meeting are > going to take place on our Trello board: > > https://trello.com/b/WbqPNl2S/avocado > > See you there. > > Best Regards, > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 54.0: House of Cards
Hello everyone, This is yet another Avocado release announcement: 54.0 is now available! Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/54_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/54.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 26 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 54.1: House of Cards (minor release)
Hello everyone, Right on the heels of the 54.0 release, the Avocado team would like to apologize for a mistake that made into that version. The following change, as documented on 54.0 has been **reverted** on this 54.1 release. * Test ID format Avocado has been using for a while received a minor tweak, to allow for better serialization into some filesystem types, such as Microsoft Windows' ones. Basically, the character that precedes the variant name, a separator, used to be ``;``, which is not allowed on some filesystems. Now, a ``+`` character is used. A Test ID ``sleeptest.py:SleepTest.test;short-beaf`` on a previous Avocado version is now ``sleeptest.py:SleepTest.test+short-beaf``. The reason for the revert and the new release, is that the actual character causing trouble in Windows filesystems was "lost in translation". The culprit was the ``:`` character, and not ``;``. This means that the Variant ID separator character change was unnecessary, and another fix is necessary. Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/54_1.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/54.1/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 26 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado RPM repository / Avocado-VT assets down
Hi everyone, The Avocado RPM repository and the Avocado-VT asset servers are currently down. We're working to reestablish them ASAP. In the mean time, if you need access to Avocado's official packages, there's a temporary repository definitions at: http://18.222.0.242/avocado-el.repo http://18.222.0.242/avocado-fedora.repo For Enterprise Linux and Fedora, respectively. For the Avocado-VT assets, the bootstrap process should work around the unavailability of the server. Sorry about any inconvenience this downtime has caused. -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Need help for avocado-vt config file
On 10/03/2017 10:21 AM, ChandraShekar Shastri wrote: > Hi, > > I have cloned the avocado and avocado-vt from the git on ubuntu. > > When I did the bootstrap using libvirt, it doesn't create the shared > folder under the /var/lib/.* > > I know the path can be changed and also it does created the directory > structure of all the guest-os under > the /usr/share/avocado-plugins-vt/shared/.* > > and does automatically generate the guest-os.cfg at bootstrap. > > But this is a huge list, and also the to run the specific tests and > customize my guest-os I would like to come up with simple config file. > > /var/lib/avocado/data/avocado-vt# ls > backends downloads gpg images isos steps_data test-providers.d > > > Any help on this would be appreciated. Hi Chandrashekar, I find it hard to provide you with any answer, because you have not really asked any specific question. Did you find an issue with "avocado vt-bootstrap --vt-type libvirt" on Ubuntu? If so, would you care to go through the exact steps, including the Avocado version, the commands you executed, and the output you got versus what you expected? Assuming that was an issue, after we fix that, we can continue unto other possible issues. Regards, - Cleber. > > Thanks, > Chandrashekar > > > On Sat, Sep 30, 2017 at 5:24 PM, ChandraShekar Shastri > mailto:pm.chandrashe...@gmail.com>> wrote: > > Hi, > > I am working on the avocado-vt on the aarch64 architecture and > struggling to build one simple config to run some 10+ test cases. > > Could you please help me on this. > > I know I should be directly writing an email like this and I am > sorry about, but really having nightmare to get config file which > works for arm. > > Thanks, > Chandrashekar > > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
Re: [Avocado-devel] Avocado RPM repository / Avocado-VT assets down
Hi everyone, After a long downtime, the servers that host both RPM repo and Avocado-VT assets are back up. Thanks for all the patience, and sorry again about the inconvenience. - Cleber. On 10/02/2017 10:52 PM, Cleber Rosa wrote: > Hi everyone, > > The Avocado RPM repository and the Avocado-VT asset servers are > currently down. We're working to reestablish them ASAP. > > In the mean time, if you need access to Avocado's official packages, > there's a temporary repository definitions at: > > http://18.222.0.242/avocado-el.repo > http://18.222.0.242/avocado-fedora.repo > > For Enterprise Linux and Fedora, respectively. For the Avocado-VT > assets, the bootstrap process should work around the unavailability of > the server. > > Sorry about any inconvenience this downtime has caused. > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] Avocado release 55.0: Never Let Me Go
Hello everyone, This is yet another Avocado release announcement: 55.0 is now available! Release Notes = Since we host the release notes alongside our official documentation, please refer to the following link for the complete information about this release: http://avocado-framework.readthedocs.io/en/latest/release_notes/55_0.html Installing Avocado == Instructions are available in our documentation on how to install either with packages or from source: http://avocado-framework.readthedocs.io/en/55.0/GetStartedGuide.html#installing-avocado Updated RPM packages are available for EPEL 7, Fedora 26 and 25. Happy hacking and testing! -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] signature.asc Description: OpenPGP digital signature
[Avocado-devel] RFC: avocado.utils.process (or, how to handle the split stdout/stderr recording)
Hi everyone,
This is kind of brainstorm about the really annoying issue we've trying
to deal with in Avocado. The root problem we're trying to fix is that
Avocado's "output check" feature can not be used when the test's
generated content combines both stdout and stderr in a single file. One
example is QEMU's iotests[1].
The root cause of this limitation is that avocado.utils.process.run()[2]
(because of avocado.utils.process.SubProcess[3]) decided a long time ago
to split the content of a process STDOUT and STDERR to separate files
(and different logging streams). This is reflected on Avocado's "output
check" feature[4]. Users can choose to record the test's generated
STDOUT, STDERR, both (all) or none of them. This is how the command
line options look like:
--output-check-record {none,all,stdout,stderr}
Record output streams of your tests to reference files
(valid options: none (do not record output streams),
all (record both stdout and stderr), stdout (record
only stderr), stderr (record only stderr). Current:
none
Which map to the following recorded files:
+--+++
| OPTION | RECORDED FILES | CONTENT|
+--+++
| none |||
+--+++
| | stdout | process FD 1 |
+ all +++
| | stderr | process FD 2 |
+--+++
| stdout | stdout | process FD 1 |
+--+++
| stderr | stderr | process FD 2 |
+--+++
Notice how the "all" option still creates two separate files, with
content coming from individual file descriptors. Adding another record
option would actually be pretty simple, that is, something like:
+--+++
| OPTION | RECORDED FILES | CONTENT|
+--+++
| combined | combined_stdout_stderr | process FD 1 and 2 |
+--+++
What is *not* easily possible, is to have two options such as "all" and
"combined" at the same time. The reason is that it the order of the
content written to "combined_stdout_stderr" can not be guaranteed to be
correct.
I've attempted to allow for both options to be used at once, by using a
pair of file-like objects with specialized writes that would either
share single list of records or just share a secondary file with a lock
(pseudo/prototype code):
class TeeFile(file):
def __init__(self, path, tee_path, tee_lock, mode='a'):
super(TeeFile, self).__init__(path, mode)
self._tee_path = tee_path
self._tee_lock = tee_lock
def write(self, record):
self.tee_lock.acquire()
super(TeeFile, self).write(record)
with open(self._tee_path, 'a') as tee_file:
tee_file.write(record)
tee_file.flush()
self.tee_lock.release()
But the fact that the Python standard library "subprocess.Popen"
implementation works with file descriptors doesn't make it possible to
implement a shared/synchronized write strategy. Rewriting parts of
"subprocess.Popen" or "avocado.utils.process.SubProcess" is, IMO, a
clear sign to not attempt something way more complicated and error prone
for the sake of this feature.
Accepting the fact that either combined (stdout+stderr) or a split
(stdout, stderr) will be available, the point of option names comes into
play. Both "stdout", "stderr" and "none" are aptly named. But, with
the introduction of a third *real* output record option/file, "all"
becomes a rather bad option name.
To keep things simple, I'd add an alias from "all" to "both",
deprecating the former. The new option would be named either "combined"
and the generated file named "combined_stdout_stderr" by default. It's
a rather verbose name, but I'd go with explicit option names rather than
confusing ones.
Comments?
--
[1] -
https://git.qemu.org/?p=qemu.git;a=blob;f=tests/qemu-iotests/check;h=e6b6ff7a047562099c2fb5f0fd35652780c9c006;hb=HEAD#l757
[2] -
http://avocado-framework.readthedocs.io/en/55.0/api/utils/avocado.utils.html#avocado.utils.process.run
[3] -
http://avocado-framework.readthedocs.io/en/55.0/api/utils/avocado.utils.html#avoc
