El mar, 25-07-2017 a las 22:59 +1000, Michael Palimaka escribió: > On 07/25/2017 07:22 AM, Sergei Trofimovich wrote: > > 2. Q: How to make arch testing faster and easier? > > > > A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing > > required" will be automatically tested and keyworded. > > > > [handwave] automated tinderbox setup would help a lot > > to now upfront what fails to built, fails tests. > > I've had similar thoughts about this and have already been working on > some tooling for this. > > We would need to establish exactly what criteria must be met for an > automated test to be considered as successful. > > Here's a sample report that my tool produces: > https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df01 > 7e14-bd68-47e2-9738-554e7ba1cf10.html > > In this case, would it be enough that it builds and tests pass? What > about the QA issues?
Personally, I don't feel QA issues as major enough to block a stabilization, usually they won't cause major issues for stable users and, if they do, for sure they shouldn't be only QA issues :/ > Do we need someone to review them to determine if > they should block stabilisation, or if they're even a regression or not? > Regarding general regressions, that is probably the harder point to handle automatically. In the past, the scripts in arch-tools.git were avoiding to open automated stable bug reports for packages having opened bugs (excluding enhancement and QA bug reports), but that approach is not too good as, for example, having a bug report asking for a version bump but tagged as "normal" and not "enhancement" will lead to the bug not being opened :S Then, I am unsure about if that part can be done automatically :/
