On 09/17/2014 09:53 AM, Ricardo Kirkner wrote: > > > On Wed, Sep 17, 2014 at 11:48 AM, Jamie Strandboge <[email protected] > <mailto:[email protected]>> wrote: > > On 09/17/2014 09:31 AM, Jamie Strandboge wrote: > ... > > >> All of this sounds pretty reasonable, however I'm now somewhat > confused. I think > >> we're talking about different things here. So to disambiguate, > >> > >> 1. the process is to override errors so to not automatically reject, > but force a > >> manual review, and *not* to override errors to allow an automatic > approval, right? > > > > What you have implemented thus far is perfect. What Daniel is wanting > to do is > > further reduce the number of manual reviews for apps where the app was > already > > manually reviewed, so it doesn't have to be next time. What I have > discussed > > allows for the overrides to allow warnings and errors that would > normally prompt > > a manual review to be ignored. I also discussed that we could have > logic around > > this so that sometimes errors could get a pass, but other times not and > if not, > > there is extra information that is available for display to the > reviewer. > > > I reread your previous email and see where the confusion may lie. The > click > reviewers team is used to "manually reviewing" any apps that have warnings > and/or errors. This "manual review" does typically include an in depth > review. > Meanwhile, the click-reviewers-tools outputs errors and warnings and a > subset of > those errors have the 'MANUAL REVIEW' string (and now the > '"manual_review": > True' json) which indicates that the reviewer should perform an in depth > review > of the app. As such, the term "manual review" is a bit overloaded in this > conversation. > > Daniel is hoping to have an override mechanism for any errors and > warnings so > that your server logic becomes: > > 1. If there are non-overridden errors in the automated review results and > none > of them are marked as requiring manual review, the app is automatically > rejected > 2. If there are non-overridden errors in the automated review results and > some > of them are marked as requiring manual review, the app is left in the > review > queue waiting for a reviewer to pick it up and start a review (just > like it > is now) > 3. If there are errors and warnings in the automated review results and > all of > them are overridden, the app is automatically approved > 4. If there are no errors and no warnings in the automated review > results, the > app is automatically approved. > > Note-- you said only 'errors' in portions of your email (which is captured > above) which got me thinking: what does the server do when an app has no > errors > but there are warnings? > > > This sounds good. I presume the click-reviewers-tools will be marking checks > with overrides somehow in the check results json so that the server can figure > out what to do. > Right, this is what I was getting at when I said "I envision the scripts could add the applicable override json to its json output and return "pass" when overrides are applied (assuming no other warning or errors)".
> Regarding warnings, if there are warnings but no errors, the app is left in > the > review queue for a reviewer to pick it up (ie, manual review is implied) > Ok, cool. -- Jamie Strandboge http://www.ubuntu.com/
signature.asc
Description: OpenPGP digital signature
-- Mailing list: https://launchpad.net/~ubuntu-appstore-developers Post to : [email protected] Unsubscribe : https://launchpad.net/~ubuntu-appstore-developers More help : https://help.launchpad.net/ListHelp
