Dear Tim, On Thu, Feb 12, 2015 at 10:31:23AM +0100, Tim Flink wrote: > > [...] > > I assume, running rpmgrill in Taskotron could mean, that each rpmgrill > > test result translates to an entry in Resultsdb as an outcome (PASSED, > > FAILED). Yet the question for the developer as to why it failed is > > important. > > I don't think that desire is unique to rpmgrill and something that we > decided to leave out of the initial Taskotron system. > > My initial thought is that that a discrete state (PASS, FAIL, > NEEDS_INSPECTION etc.) a one-line-or-so summary and link to output > from rpmgrill would work well. Are there other things that you'd like > to see in terms of output? From what Kamil pointed out, there is no state missing.
> > 1. Ed Santiago is currently running RPMGrill here: > > > > http://rpmgrill-fc20.edsantiago.com:5000/recent > > > > which in terms of presenting results to the user provides a > > different experience. The results are grouped by test and provide > > more insight as to why a test failed. Test results in Resultsdb > > currently seem to have only one outcome and technical details are > > left on the build master. > > This was less of a desired final outcome for us than it was the easiest > way to get started. > > I think that our current method of having gigantic text-only log files > is far from optimal (and something that we did years ago in AutoQA > before making the logs more easily accessible) but we haven't improved > the log accessibility/readability in taskotron yet. Good to know. I guess this is something to discuss and work on? > > How does a representation like RPMGrill translate into Resultsdb? > > Are there plans/ideas on how to provide a more diversified result > > representation in Resultsdb (e.g. error reason, hint on how to solve > > an error). > > I think that having rpmgrill output static html would likely be the > best approach here unless you're planning to continue supporting the > webapp that you linked to (and from previous conversations, I'm under > the impression that was never intended to be a long-term thing). If we'd have a way to turn the TAP output into details retrievable from resultsdb, it'll be better than rpmgrill printing HTML? > > 2. Are there plans/ideas about implementing a (fulltext-) search over > > results? > > Nothing written down, no but that's been on my wishlist since before > Taskotron really got started. Unless there's more demand for the > fulltext indexing than I think there is, we'd probably start with > something less complicated like indexing the summary statements. > > I'm open to other ideas, though. Just concerned about what fulltext > indexing of all logs might entail. True. We've had quite big logs if some big packages contain a lot of regressions. Good to know that it is on the wish list. > > 3. Are there plans/ideas on implementing a waiver mechanism? > > Yeah, it'll have to happen before we start gating any > rawhide/branched/stable builds or updates for Fedora. We don't have a > design yet but it'll happen at some point. Ok. This also seems like something we might be able to contribute to. Perhaps better to discuss this in another thread? > > [1] - https://git.fedorahosted.org/git/rpmgrill.git > > > > PS: Yes - I'm aware that there is already a bitbucket repository of > > rpmgrill for Taskotron, but have only had a quick glimpse at it. > > Someone was planning to work on getting rpmgrill working in taskotron > and that bitbucket repo was meant to hold the task that's executed in > taskotron. > > From our end, that is being tracked as: > > https://phab.qadevel.cloud.fedoraproject.org/T281 > > Let us know if you have any other questions or concerns. Thanks. I'll have a look at the tasks. I don't want to make any promises, but I hope we'll find some time to get the rpmgrill tasks in a 'done' state. Thanks and Regards, -- Róman Joost Software Engineer, HSS - Software Engineering and Development (Brisbane) email: rjo...@redhat.com | tz: UTC+10 irc: #hss #rpmdiff
pgpzRdvdn6kVs.pgp
Description: PGP signature
_______________________________________________ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel