Re: Re-Scheduling Jobs for Taskotron as a User
On Thu, Apr 20, 2017 at 12:07 AM, Adam Williamson < adamw...@fedoraproject.org> wrote: > OK, like I said, half-baked =) But wdyt? > > Love it! (And I swear, it has nothing to do with the fact, that I also thought this would be a great way to solve it in a more generic manner.) ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On Tue, 2017-04-18 at 06:35 -0600, Tim Flink wrote: > One of the things that seems like it is and will be a pain point for > folks writing package-specific tasks is how to work through the times > when there was an issue in the task and things didn't run well. At the > moment, the only solution we have is to re-build the affected package > or to pester an admin to re-run the trigger - neither of which is an > ideal answer. > > I was thinking about how to improve this in the near future and am > wondering about adding a "reschedule" button to execdb jobs: > > 1. authenticated user clicks on "reschedule" button > 2. execdb makes an api call to the buildmaster to find the parameters > which were used for that task > 3. using the data from 2), execdb starts a new job for just that > item and item type > > I'm not thrilled at the idea of code duplication here between trigger > and execdb but compared to figuring out a web interface for trigger, > this seems like a more tractable solution for the short/medium term. > > Thoughts? Here's a Half-Baked Idea (tm): What about a thing that's basically a simple web service that allows FAS-authenticated people to trigger the sending of templated fedmsgs? This could be used in various workflows like this. For this case, we'd implement a fedmsg template which would basically say 'authenticated user X asks that you please re-run test Y in test system Z'. Then we could plug little bits of code into Bodhi, execdb, really anything else you like, which would provide a button which, when clicked, calls out to that web service to send the 'restart test' fedmsg. The test dispatchers for Taskotron, openQA etc. would then just need to listen out for such fedmsgs, and do whatever they felt appropriate: you could implement policy at the dispatcher level - ignore requests from users without certain attributes (or even stash them for manual review, or something), accept requests from other users with different priorities, stuff like that. For feedback...well, if the test *did* get restarted, a fedmsg will naturally be emitted which the requesting system could listen out for, and if the scheduler decided to reject or sideline the request, it could emit a fedmsg saying it had done so. OK, like I said, half-baked =) But wdyt? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
I'd like to see something like: $ fedpkg test Unfortunately, internally we are using "$ rhpkg tests" to clone the test suite into the repo Vít Dne 18.4.2017 v 14:35 Tim Flink napsal(a): > One of the things that seems like it is and will be a pain point for > folks writing package-specific tasks is how to work through the times > when there was an issue in the task and things didn't run well. At the > moment, the only solution we have is to re-build the affected package > or to pester an admin to re-run the trigger - neither of which is an > ideal answer. > > I was thinking about how to improve this in the near future and am > wondering about adding a "reschedule" button to execdb jobs: > > 1. authenticated user clicks on "reschedule" button > 2. execdb makes an api call to the buildmaster to find the parameters > which were used for that task > 3. using the data from 2), execdb starts a new job for just that > item and item type > > I'm not thrilled at the idea of code duplication here between trigger > and execdb but compared to figuring out a web interface for trigger, > this seems like a more tractable solution for the short/medium term. > > Thoughts? > > Tim > > > ___ > qa-devel mailing list -- qa-devel@lists.fedoraproject.org > To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org signature.asc Description: OpenPGP digital signature ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On Tue, 2017-04-18 at 10:18 -0600, Kevin Fenzi wrote: > On 04/18/2017 10:08 AM, Adam Williamson wrote: > > > One thought: we actually have the same problem for openQA, now it's > > running tests on updates. This isn't very visible at present so devs > > haven't been asking about it, but I suspect as soon as the results show > > up in Bodhi, someone will be wanting a 're-run test' button. I'm not > > sure how far we could share any part of implementing it, though... > > So what is currently triggering the tests now? > fedmsgs when builds are tagged into specific koji tags? > > I know it's not a fancy gui, but could we just untag/retag something to > emit another fedmsg to retest? Dunno about Taskotron package-level tests. openQA update-level tests trigger on 'bodhi.update.edit' and 'bodhi.update.request.testing' messages; you can trigger a re-run of *all* the tests by editing the update, but that's a bit silly. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On 04/18/2017 10:08 AM, Adam Williamson wrote: > One thought: we actually have the same problem for openQA, now it's > running tests on updates. This isn't very visible at present so devs > haven't been asking about it, but I suspect as soon as the results show > up in Bodhi, someone will be wanting a 're-run test' button. I'm not > sure how far we could share any part of implementing it, though... So what is currently triggering the tests now? fedmsgs when builds are tagged into specific koji tags? I know it's not a fancy gui, but could we just untag/retag something to emit another fedmsg to retest? kevin signature.asc Description: OpenPGP digital signature ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On Tue, 2017-04-18 at 06:35 -0600, Tim Flink wrote: > One of the things that seems like it is and will be a pain point for > folks writing package-specific tasks is how to work through the times > when there was an issue in the task and things didn't run well. At the > moment, the only solution we have is to re-build the affected package > or to pester an admin to re-run the trigger - neither of which is an > ideal answer. > > I was thinking about how to improve this in the near future and am > wondering about adding a "reschedule" button to execdb jobs: > > 1. authenticated user clicks on "reschedule" button > 2. execdb makes an api call to the buildmaster to find the parameters > which were used for that task > 3. using the data from 2), execdb starts a new job for just that > item and item type > > I'm not thrilled at the idea of code duplication here between trigger > and execdb but compared to figuring out a web interface for trigger, > this seems like a more tractable solution for the short/medium term. > > Thoughts? One thought: we actually have the same problem for openQA, now it's running tests on updates. This isn't very visible at present so devs haven't been asking about it, but I suspect as soon as the results show up in Bodhi, someone will be wanting a 're-run test' button. I'm not sure how far we could share any part of implementing it, though... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On Tue, 18 Apr 2017 14:53:07 +0200 Kamil Paral wrote: > On Tue, Apr 18, 2017 at 2:35 PM, Tim Flink wrote: > > > One of the things that seems like it is and will be a pain point for > > folks writing package-specific tasks is how to work through the > > times when there was an issue in the task and things didn't run > > well. At the moment, the only solution we have is to re-build the > > affected package or to pester an admin to re-run the trigger - > > neither of which is an ideal answer. > > > > I was thinking about how to improve this in the near future and am > > wondering about adding a "reschedule" button to execdb jobs: > > > > Execdb web is not searchable, people will never find their job. It > makes a lot of sense to have the button in execdb, because that's > where we want to show people crashed jobs (those will not appear > neither in resultsdb nor in Bodhi). So thumbs up to that idea. But > the search seems to be a prerequisite to this. Or we need to make > sure crashed jobs are sent out with direct links to execdb to the > relevant maintainers, if that turns out to be easier. Yeah, it's not an ideal solution but in my mind, it's better than asking folks to rebuild just to get the task re-run. > > 1. authenticated user clicks on "reschedule" button > > > > Authentication itself sounds like a non-trivial issue. I wonder if we > can avoid it at least temporarily. Maybe allow anyone to click the > button, but limit the number of times it can be pressed (e.g. max 5 > times)? Also introduce a timeout after the press (max once per hour > since the last re-run), and don't allow to re-run old jobs (e.g. > older than 24 hours)? > > Maybe authentication will be easier after all :) Maybe we don't even > need to check package permissions in the beginning, just making sure > the person has a valid FAS (cla+1) account. I was hoping that we'd be able to auth against ipsilon without much added effort. Ideally we'd restrict things to someone with ACLs to the involved package but that's far from required. cla+1 should be good enough for now, I think. Limiting it to packagers could also be reasonable but I'm not sure it's going to be a big deal so long as we can keep track of who's doing the reschedules through at least a log file. Tim > > > 2. execdb makes an api call to the buildmaster to find the > > parameters which were used for that task > > 3. using the data from 2), execdb starts a new job for just that > > item and item type > > > > I'm not thrilled at the idea of code duplication here between > > trigger and execdb but compared to figuring out a web interface for > > trigger, this seems like a more tractable solution for the > > short/medium term. > > > > Thoughts? > > > > Tim > > > > ___ > > qa-devel mailing list -- qa-devel@lists.fedoraproject.org > > To unsubscribe send an email to > > qa-devel-le...@lists.fedoraproject.org > > > > pgpQbLfl5KjHY.pgp Description: OpenPGP digital signature ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re: Re-Scheduling Jobs for Taskotron as a User
On Tue, Apr 18, 2017 at 2:35 PM, Tim Flink wrote: > One of the things that seems like it is and will be a pain point for > folks writing package-specific tasks is how to work through the times > when there was an issue in the task and things didn't run well. At the > moment, the only solution we have is to re-build the affected package > or to pester an admin to re-run the trigger - neither of which is an > ideal answer. > > I was thinking about how to improve this in the near future and am > wondering about adding a "reschedule" button to execdb jobs: > Execdb web is not searchable, people will never find their job. It makes a lot of sense to have the button in execdb, because that's where we want to show people crashed jobs (those will not appear neither in resultsdb nor in Bodhi). So thumbs up to that idea. But the search seems to be a prerequisite to this. Or we need to make sure crashed jobs are sent out with direct links to execdb to the relevant maintainers, if that turns out to be easier. > 1. authenticated user clicks on "reschedule" button > Authentication itself sounds like a non-trivial issue. I wonder if we can avoid it at least temporarily. Maybe allow anyone to click the button, but limit the number of times it can be pressed (e.g. max 5 times)? Also introduce a timeout after the press (max once per hour since the last re-run), and don't allow to re-run old jobs (e.g. older than 24 hours)? Maybe authentication will be easier after all :) Maybe we don't even need to check package permissions in the beginning, just making sure the person has a valid FAS (cla+1) account. > 2. execdb makes an api call to the buildmaster to find the parameters > which were used for that task > 3. using the data from 2), execdb starts a new job for just that > item and item type > > I'm not thrilled at the idea of code duplication here between trigger > and execdb but compared to figuring out a web interface for trigger, > this seems like a more tractable solution for the short/medium term. > > Thoughts? > > Tim > > ___ > qa-devel mailing list -- qa-devel@lists.fedoraproject.org > To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org > > ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
Re-Scheduling Jobs for Taskotron as a User
One of the things that seems like it is and will be a pain point for folks writing package-specific tasks is how to work through the times when there was an issue in the task and things didn't run well. At the moment, the only solution we have is to re-build the affected package or to pester an admin to re-run the trigger - neither of which is an ideal answer. I was thinking about how to improve this in the near future and am wondering about adding a "reschedule" button to execdb jobs: 1. authenticated user clicks on "reschedule" button 2. execdb makes an api call to the buildmaster to find the parameters which were used for that task 3. using the data from 2), execdb starts a new job for just that item and item type I'm not thrilled at the idea of code duplication here between trigger and execdb but compared to figuring out a web interface for trigger, this seems like a more tractable solution for the short/medium term. Thoughts? Tim pgpaqRUCMC5uL.pgp Description: OpenPGP digital signature ___ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org