On 11/5/20 11:30 AM, Milan Zamazal wrote:
Marcin Sobczyk <msobc...@redhat.com> writes:
On 11/4/20 11:29 AM, Yedidyah Bar David wrote:
Perhaps what you want, some day, is for the individual tests to have make-style
dependencies? So you'll issue just a single test, and OST will only
run the bare minimum for running it.
Yeah, I had the same idea. It's not easy to implement it though.
'pytest' has a "tests are independent" design, so we would need to
build something on top of that (or try to invent our own test
framework, which is a very bad idea). But even with a
dependency-resolving solution, there are tests that set something up
just to bring it down in a moment (by design), so we'd probably need
some kind of "provides" and "tears down" markers. Then you have the
fact that some things take a lot of time and we do other stuff in
between, while waiting - dependency resolving could force things to
happen linearly and the run times could skyrocket... It's a complex
subject that requires a serious think-through.
Actually I was once thinking about introducing test dependencies in
order to run independent tests in parallel and to speed up OST runs this
way. The idea was that OST just waits on something at many places and
it could run other tests in the meantime (we do some test interleaving
in extreme cases but it's suboptimal and difficult to maintain).
Yeah, I think I remember you did that during one of OST's hackathons.
When arranging some things manually, I could achieve a significant
speedup. But the problem is, of course, how to make an automated
dependency management and handle all the possible situations and corner
cases. It would be quite a lot of work, I think.
Exactly. I.e. I can see there's [1], but of course that will work only
on py3.
The dependency management is something we'd have to implement and maintain
on our own probably.
Then of course we'd be introducing the test repeatability
problem, since ordering of things for different runs might be different,
which in current state of OST is something I'd like to avoid.
[1] https://pypi.org/project/pytest-asyncio/
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QLPDB4VI2P3WZ7SHQMWUS6LSMA5SVAA7/
