So the script that actually runs the browser test is not in a generic useful form but it is: https://gist.github.com/jdlrobson/32b607f8009e897ee80c
It uses the GerritCommandLine tool to do grabbing and reviewing https://github.com/jdlrobson/GerritCommandLine Ideally if we can use labs-tools-gerrit-to-redis for identifying patches and then pulling them down we wouldn't need the GerritCommandLine tool since the code to submit a review is pretty trivial and captured in this function: https://github.com/jdlrobson/GerritCommandLine/blob/master/gerrit.py#L277 I've also put this in the task https://phabricator.wikimedia.org/T101069#1379462 We'll probably want an instance per extension, to simplify having to worry about dependencies (we can just run a git update on all extensions after each checkout) On Thu, Jun 18, 2015 at 4:32 AM, Joaquin Oltra Hernandez <jhernan...@wikimedia.org> wrote: > Awesome Jon! I'm so happy to finally see this developing :DD > > Loving the : `Browserbot happy!` > > I've noticed it can report either the name of the failing test or the full > log. What do you think if we show that, and a url with the pasted log > somewhere publicly to not put too much noise on the comments but still be > able to see it? Something like https://phabricator.wikimedia.org/paste/... > > +1 to where is the source. > +1 to documenting how you've set it all up on wiki somewhere. > > I also think we need a catchy phrase for the -1s! > > Thanks for you work on this, we'll get more focused time for it soon. > > On Thu, Jun 18, 2015 at 11:33 AM, Sam Smith <samsm...@wikimedia.org> wrote: >> >> I agree with Florian everything that you've written should be in a public >> version control system. >> >> Second, I'd ask that you document your experiences so far in getting this >> set up and how it works so that other members of the vertical can help to >> maintain it moving forward. >> >> Third, great work!!1 >> >> <3 >> >> –Sam >> >> On Thu, Jun 18, 2015 at 7:09 AM, florian.schmidt.wel...@t-online.de >> <florian.schmidt.wel...@t-online.de> wrote: >>> >>> >>> > It's currently working via a script that you can find here: >>> > /srv/mediawiki/extensions/Gather/tests/browser/Barry.sh >>> >>> It would be great to have the script in a public version control system >>> (e.g. github?), especially for people, e.g. volunteers, who can't ssh to >>> gather-browser-tests.eqiad.wmflabs[1] >>> >>> [1] all people, who're not members of >>> https://wikitech.wikimedia.org/wiki/Nova_Resource:Mobile-smoketests >>> >>> Best, >>> Florian >>> >>> -----Original-Nachricht----- >>> Betreff: [WikimediaMobile] [Update] Browser tests per patch >>> Datum: Thu, 18 Jun 2015 03:27:32 +0200 >>> Von: Jon Robson <jrob...@wikimedia.org> >>> An: "QA (software quality assurance) for Wikimedia projects." >>> <q...@lists.wikimedia.org>, mobile-l <mobile-l@lists.wikimedia.org> >>> >>> Background: mobile wants to gain more confidence in its browser tests >>> by running a subset of browser tests on a case by case basis [0]. >>> >>> Good news: I've got a proof of concept running and Barry the browser >>> test bot has given some legitimate helpful reviews to Gather [1]. >>> >>> Even better news: It's proving itself valuable already [2]. >>> As you can see in the messages the bot has posted on [3] we have a >>> couple of options on display option format for his reviews. >>> >>> So.. hopefully this short experience has sold you all already. >>> >>> This script is currently a manual job and needs a bit of tweaking >>> before we can put it in a cron job/run it always - it needs to watch >>> for new commits and then run a modification of the above script on a >>> per case basis (if two versions of it run in parallel we have an >>> issue). >>> >>> Definitely something we should push for next sprint! >>> >>> Long live Barry bot! >>> >>> Devs... (everyone else now of what follows is likely to be useful): >>> I got the labs instance up and running on: >>> http://gather-browser-tests.wmflabs.org/wiki/Main_Page >>> >>> Most of you in readership team should be able to ssh >>> gather-browser-tests.eqiad.wmflabs >>> Let me know if you have no access. >>> >>> It's currently working via a script that you can find here: >>> /srv/mediawiki/extensions/Gather/tests/browser/Barry.sh >>> >>> [0] https://phabricator.wikimedia.org/T100293 >>> [1] >>> https://gerrit.wikimedia.org/r/#/q/reviewer:jdlrobson%252Bbarry%2540gmail.com+status:open,n,z >>> [2] https://gerrit.wikimedia.org/r/#/c/218731/ >>> >>> _______________________________________________ >>> Mobile-l mailing list >>> Mobile-l@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Mobile-l mailing list >>> Mobile-l@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> >> >> >> _______________________________________________ >> Mobile-l mailing list >> Mobile-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> > > > _______________________________________________ > Mobile-l mailing list > Mobile-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/mobile-l > -- Jon Robson * http://jonrobson.me.uk * https://www.facebook.com/jonrobson * @rakugojon _______________________________________________ Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
