On Tue, Jun 28, 2022 at 14:05 Ani Sinha <a...@anisinha.ca> wrote: > On Tue, Jun 28, 2022 at 1:58 PM Thomas Huth <th...@redhat.com> wrote: > > > > On 28/06/2022 10.23, Daniel P. Berrangé wrote: > > > On Tue, Jun 28, 2022 at 01:21:35PM +0530, Ani Sinha wrote: > > >> On Tue, Jun 28, 2022 at 1:19 PM Daniel P. Berrangé < > berra...@redhat.com> wrote: > > >>> > > >>> On Tue, Jun 28, 2022 at 09:25:35AM +0200, Thomas Huth wrote: > > >>>> On 28/06/2022 09.10, Michael S. Tsirkin wrote: > > >>>>> On Tue, Jun 28, 2022 at 09:03:33AM +0200, Thomas Huth wrote: > > >>>>>>>>>>>> No problem with that. So that's venv. But do we need pip > and pulling > > >>>>>>>>>>>> packages from the net during testing? > > >>>>>>>>>>> > > >>>>>>>>>>> We do that too. See requirements.txt in tests/ > > >>>>>>>>>>> Following two are downloaded: > > >>>>>>>>>>> avocado-framework==88.1 > > >>>>>>>>>>> pycdlib==1.11.0 > > >>>>>>>>>>> > > >>>>>>>>>>> Also see this line in Makefie.include: > > >>>>>>>>>>> > > >>>>>>>>>>> $(call quiet-venv-pip,install -r $(TESTS_VENV_REQ)) > > >>>>>>>>>> > > >>>>>>>>>> Right but that's avocado since it pulls lots of stuff from > > >>>>>>>>>> the net anyway. > > >>>>>>>>>> Are the libraries in question not packaged on major distros? > > >>>>>>>>> > > >>>>>>>>> Currently I only need this: > > >>>>>>>>> https://github.com/python-tap/tappy > > >>>>>>>>> which is the basic TAP processing library for python. > > >>>>>>>>> > > >>>>>>>>> It seems its only installed through pip: > > >>>>>>>>> https://tappy.readthedocs.io/en/latest/ > > >>>>>>>>> > > >>>>>>>>> I do not think this is packaged by default. It's such a basic > library > > >>>>>>>>> for parsing test output that maybe we can keep this somewhere > within > > >>>>>>>>> the python src tree? Not sure ... > > >>>>>>>> > > >>>>>>>> It's pretty small for sure. Another submodule? > > >>>>>>> > > >>>>>>> Unlike BITS, this one is likely going to be maintained for a > while and > > >>>>>>> will receive new releases through > > >>>>>>> https://pypi.org/project/tap.py/ > > >>>>>>> so forking is OK but someone has to keep this updated. > > >>>>>>> > > >>>>>>> I am open to anything. Whatever feels right is fine to me. > > >>>>>> > > >>>>>> John Snow is currently working on the "Pythonification" of > various QEMU > > >>>>>> bits, I think you should loop him into this discussion, too. > > >>>>>> > > >>>>>> Thomas > > >>>>> > > >>>>> submodule does not mean we fork necessarily. We could have > > >>>>> all options: check for the module and use it if there, if not > > >>>>> use one from system if not there install with pip .. > > >>>>> But yea, I'm not sure what's best either. > > >>>> > > >>>> submodules create a dependency on an internet connection, too. So > before you > > >>>> add yet another submodule (which have a couple of other > disadvantages), I > > >>>> think you could also directly use the venv here. > > >>> > > >>> Definitely not submodules. > > >>> > > >>> We need to get out of the mindset that submodules are needed for > every new > > >>> dependancy we add. Submodules are only appropriate if the external > project > > >>> is designed to be used as a copylib (eg the keycodemapdb tool), or > if we > > >>> need to bundle in order to prevent a regression for previously > deployed > > >>> QEMU installs where the dependancy is known not to exist on all our > > >>> supported platforms. > > >>> > > >>> This does not apply in this case, because the proposed use of tappy > is > > >>> merely for a test case. Meson just needs to check if tappy exists > and if > > >>> it does, then use it, otherwise skip the tests that need it. The > user can > > >>> arrange to install tappy, as they do with the majority of other deps. > > >>> > > >>> If John's venv stuff is relevant, then we don't even need the meson > checks, > > >>> just delegate to the venv setup. > > >>> > > >>> Regardless, no submodules are needed or desirable. > > >> > > >> What about keeping biosbits stuff? Source or pre-built. > > > > > > Shipping them as pre-built binaries in QEMU is not a viable option > > > IMHO, especially for grub as a GPL'd project we need to be extremely > > > clear about the exact corresponding source and build process for any > > > binary. > > > > > > For this kind of thing I would generally expect the distro to provide > > > packages that we consume. Looking at biosbits I see it is itself > > > bundling a bunch more 3rd party projects, libffi, grub2, and including > > > even an ancient version of python as a submodule. > > > > > > So bundling a pre-built biosbits in QEMU appears to mean that we're in > > > turn going to unexpectedly bundle a bunch of other 3rd party projects > > > too, all with dubious license compliance. I don't think this looks like > > > something we should have in qemu.git or qemu tarballs. It will also > > > make it challenging for the distro to take biosbits at all, unless > > > those 3rd party bundles can be eliminated in favour of using existing > > > builds their have packaged for grub, python, libffi, etc. > > > > So if this depends on some third party binary bits, I think this is > pretty > > similar to the tests in the avocado directory ... there we download third > > party binaries, too... Wouldn't it make sense to adapt your tests to that > > framework? > > I do not want to bring in the whole avocado framework because it would > unnecessarily make things complicated. I just need the qemu machine > python library and that is enough. For downloading third party stuff,
we can simply wget things from somewhere. https://pypi.org/project/wget/ That get_asset() call is an overkill for downloading two archives. > > > > > Thomas > > >
