Re: [QA] Quality-Assurance phab project

2015-03-25 Thread Chris McMahon
I think that tells us two things (not mutually exclusive): 1) We need more non-RelEng engineers watching their browser test builds and reporting bugs. 2) We already do have engineers watching their browser test builds and reporting bugs, they just don't use #browser-test-bug. I don't claim

[QA] Current status of Selenium and Chrome

2015-03-10 Thread Chris McMahon
Hi QAers, I just wanted to let you know that in case you are experiencing unexpected failures in browser tests for Chrome, there seems to be Selenium bug in SauceLabs affecting how Selenium invokes WMF-style overlays. Affected repos are MobileFrontend, Echo, VisualEditor, and possibly others. I

Re: [QA] Investigate logging common JavaScript exceptions

2015-02-25 Thread Chris McMahon
FWIW, you might want to read Jim Evans on the subject: http://jimevansmusic.blogspot.com/2013/09/capturing-javascript-errors-in.html On Wed, Feb 25, 2015 at 3:39 PM, Gergo Tisza gti...@wikimedia.org wrote: On Tue, Feb 24, 2015 at 2:53 AM, Sam Smith samsmith at wikimedia.org I'm working on

Re: [QA] Investigate logging common JavaScript exceptions

2015-02-24 Thread Chris McMahon
By design, Selenium knows nothing about javascript errors. We don't really have any tools to hand to do this sort of thing, the approach that Sam talks about makes more sense. -Chris On Tue, Feb 24, 2015 at 3:14 PM, Jon Robson jrob...@wikimedia.org wrote: I agree that this probably needs

[QA] New user story

2015-02-05 Thread Chris McMahon
http://tharys.blogspot.com/2015/02/cucumber-browser-testing-with-mediawiki.html This person commented on something I said on Twitter. I sent them a link to https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing. One day later they put up the blog post above. Starting from nothing, with

[QA] watir-webdriver change

2015-01-16 Thread Chris McMahon
Just FYI, Jari Bakken is stepping down as the maintainer of watir-webdriver and selenium-webdriver. This won't affect us immediately, but we should be aware of it: https://groups.google.com/forum/#!topic/selenium-developers/h2Ie4FNHmq4 ___ QA mailing

[QA] updated password for Selenium_user on test2wiki

2015-01-13 Thread Chris McMahon
I changed the password for MEDIAWIKI_PASSWORD_SELENIUM_USER_WIKIPEDIA_ORG. Let me know if you need it and can't find it. ___ QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa

Re: [QA] Zuul / jenkins is broken

2015-01-05 Thread Chris McMahon
In Jenkins I clicked prepare for shutdown, then cancelled the operation, hoping to unstick jenkins. I saw the beta-scap-eqiad job run after that. Then I disabled/enabled gearman I made a trivial update to a gerrit patch but did not see zuul pick up the change. I seem to recall that at one point

Re: [QA] Fwd: [Labs-l] Filesystem downtime to schedule

2014-12-31 Thread Chris McMahon
On Wed, Dec 31, 2014 at 10:48 AM, Bryan Davis bd...@wikimedia.org wrote: I think this will mess some things up in beta (file uploads, debug logging). If having a partial outage of beta from just after a group0 deploy to just before a new branch release sounds bad to any of you you may want to

Re: [QA] [WikimediaMobile] big improvements to the MobileFrontend browser test repo

2014-12-05 Thread Chris McMahon
On Fri, Dec 5, 2014 at 2:23 AM, Joaquin Oltra Hernandez jhernan...@wikimedia.org wrote: Here you can see vagrant roles, final output of the tests and env variables https://gist.github.com/joakin/eae3b086dd5a0c498669 I'm running the tests with phantomjs by default for convenience and speed,

Re: [QA] [WikimediaMobile] big improvements to the MobileFrontend browser test repo

2014-12-05 Thread Chris McMahon
On Fri, Dec 5, 2014 at 8:17 AM, Chris McMahon cmcma...@wikimedia.org wrote: On Fri, Dec 5, 2014 at 2:23 AM, Joaquin Oltra Hernandez jhernan...@wikimedia.org wrote: Here you can see vagrant roles, final output of the tests and env variables https://gist.github.com/joakin

Re: [QA] Reg::New issue search in Phabricator

2014-12-04 Thread Chris McMahon
Then step to use RSpec3 expect syntax On Tue, Dec 2, 2014 at 4:21 PM, Chris McMahon cmcma...@wikimedia.org wrote: Hi Jagori, Some of these repos may pose problems. For example, ArticleFeedbackv5 is dead code and the feature itself is no longer enabled in any test environments. Here

[QA] big improvements to the MobileFrontend browser test repo

2014-12-04 Thread Chris McMahon
Hi mobile folk, After 48 patches merged, I have updated all the browser tests in the MobileFrontend repo to conform to RSpec3 syntax. Along the way I did a few other things: * removed every sleep() statement except one necessary to get around a bug in Chrome * consolidated the lines within each

Re: [QA] Reg::New issue search in Phabricator

2014-12-02 Thread Chris McMahon
let you know if I get stuck somewhere. Thanks, Jagori On Tue, Dec 2, 2014 at 2:23 AM, Chris McMahon cmcma...@wikimedia.org wrote: On Thu, Nov 27, 2014 at 3:54 PM, jagori samajdar jagor...@gmail.com wrote: Hi Jagori, Also,would you suggest any specific repository where I should start

Re: [QA] Reg::New issue search in Phabricator

2014-12-01 Thread Chris McMahon
On Thu, Nov 27, 2014 at 3:54 PM, jagori samajdar jagor...@gmail.com wrote: Hi Jagori, Also,would you suggest any specific repository where I should start making the rspec-expectation 'expect' syntax changes or should I just choose and start from the list :

Re: [QA] Ruby Coding Conventions and RuboCop (was: Rubocop style checker)

2014-11-13 Thread Chris McMahon
I have an issue with Metrics/LineLength: Max: 128 In a number of places we have lines longer than this that can't be shortened. -C On Thu, Nov 13, 2014 at 7:54 AM, Željko Filipin zfili...@wikimedia.org wrote: On Sat, Nov 1, 2014 at 12:05 AM, Dan Duvall dduv...@wikimedia.org wrote: Just

Re: [QA] [teampractices] Testing 101 docs from Etsy

2014-10-29 Thread Chris McMahon
On Wed, Oct 29, 2014 at 8:20 AM, Antoine Musso hashar+...@free.fr wrote: Le 29/10/2014 12:07, Quim Gil a écrit : I was wondering whether we could use Google Code-in to increase our unit testing coverage. I guess there are certain components and areas easier to to cover than others, and I

Re: [QA] Pair programming for fun and profit

2014-10-29 Thread Chris McMahon
Hello Dr. Lu... On Wed, Oct 29, 2014 at 10:55 AM, Baochuan Lu lubaoch...@gmail.com wrote: Hello everybody: My name is Baochuan Lu. I live in Bolivar (Missouri, USA CDT/UTC-6). I am lubaochuan at freenode and have a blog at http://createdtolearn.org. You might be interested in our freenode

Re: [QA] # 147 more reliable Beta Labs for QA

2014-10-29 Thread Chris McMahon
On Wed, Oct 29, 2014 at 11:31 AM, Greg Grossmeier g...@wikimedia.org wrote: Short version: Beta stability is always an on-going goal of RelEng. With that said: can you give specific examples of Beta Cluster being unstable *today*? Since you asked, the #1 issue for me right now is that beta

[QA] Testing 101 docs from Etsy

2014-10-28 Thread Chris McMahon
Relevant to our interests, this seems to two nice guides to practical unit testing in PHP from Etsy: http://codeascraft.com/2014/08/20/teaching-testing-our-testing-101-materials/ ___ QA mailing list QA@lists.wikimedia.org

Re: [QA] better page object explanation, colon confusion

2014-10-27 Thread Chris McMahon
On Fri, Oct 24, 2014 at 9:37 PM, S Page sp...@wikimedia.org wrote: E.g. https://github.com/cheezy/page-object/wiki/Elements has button(:your_name, :id = 'an_id') but our features/support/foo_page.rb files have e.g. h1(:first_heading, id: firstHeading) is this Ruby strangeness, or

Re: [QA] Software Tester.

2014-10-27 Thread Chris McMahon
Hi, On Sun, Oct 26, 2014 at 1:19 PM, Bharati Deshpande bharti...@gmail.com wrote: Hello, I would like to work as volunteer software tester and spend time looking for bugs in a Wikipedia feature under development. I would like to know more about the program and the details on how I can

[QA] nested elements in Flow tests

2014-10-25 Thread Chris McMahon
Dan pointed this out to me this week: https://github.com/cheezy/page-object/wiki/Nested-Elements It turns out we really need this feature for Flow tests. On a Flow page (or Flow board) you have a set elements within a of Topic element that repeat, and that are identifiable only in relation to

Re: [QA] Browser test guidelines and good practices

2014-10-16 Thread Chris McMahon
16, 2014 at 12:59 PM, S Page sp...@wikimedia.org wrote: On Fri, Oct 3, 2014 at 1:45 PM, Chris McMahon cmcma...@wikimedia.org wrote: I made a wiki page: https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Guidelines_and_good_practices Great, though I think it should be merged

Re: [QA] How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process

2014-10-10 Thread Chris McMahon
Me too. On Fri, Oct 10, 2014 at 10:47 AM, Dan Duvall dduv...@wikimedia.org wrote: I've just signed up as well. On Fri, Oct 10, 2014 at 4:30 AM, Željko Filipin zfili...@wikimedia.org wrote: On Fri, Oct 10, 2014 at 1:01 PM, Tobi Gritschacher tobias.gritschac...@wikimedia.de wrote:

[QA] Browser test guidelines and good practices

2014-10-03 Thread Chris McMahon
S had asked me to put together a summary of guidelines and practices as I've been refactoring my way through the Echo and Flow browser test repos. I made a wiki page: https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Guidelines_and_good_practices Comments, criticism, and updates

Re: [QA] Jenkins Performance Plugin deployed

2014-09-29 Thread Chris McMahon
This will be useful all over, thanks! -Chris On Mon, Sep 29, 2014 at 2:32 AM, Tobi Gritschacher tobias.gritschac...@wikimedia.de wrote: Hey, Antoine just deployed the Performance Plugin [1] on Jenkins. Right now only the Wikidata UI Performance Tests [2] make use of it, but I'm sure there

[QA] upstream bug reported for page_object Ruby gem

2014-09-22 Thread Chris McMahon
Hi, so I noticed last week that the Flow builds were getting deprecation warnings from the test framework saying Locating textareas with '#text_field' is deprecated. Please, use '#textarea' method instead. I don't like being deprecated, but upon updating the test, to my surprise, the

Re: [QA] upstream bug reported for page_object Ruby gem

2014-09-22 Thread Chris McMahon
, Sep 22, 2014 at 11:29 AM, Chris McMahon cmcma...@wikimedia.org wrote: Hi, so I noticed last week that the Flow builds were getting deprecation warnings from the test framework saying Locating textareas with '#text_field' is deprecated. Please, use '#textarea' method instead. I don't like

Re: [QA] Upcoming work on test environments

2014-09-18 Thread Chris McMahon
Thanks S... This indicates that it is a high priority to have separate test environments for: 1) if master works here it'll work in production on Thursday and 2) do these upcoming features work? which is what Chris McMahon describes, excerpted below. So +7 to that. More information

Re: [QA] Firefox doesn't scroll items into view in browser tests

2014-09-16 Thread Chris McMahon
I couldn't repro locally because the test failed for me. I wonder two things: one is whether or not it's a known webdriver issue: https://code.google.com/p/selenium/issues/detail?id=7249 The other is whether there is some z-axis shenanigans going on:

[QA] MMV browser tests for IE versions

2014-09-12 Thread Chris McMahon
We created a set of these, you can see them at: https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/ In the near future I would like to refine exactly what is required for these builds as regards * browser * version * OS (if applicable) * target environment (beta,

[QA] Theme for the next few weeks: Multimedia Viewer

2014-09-12 Thread Chris McMahon
Hello QAers, If you're not aware, we have a nifty new feature called the MultimediaViewer. You can see it in action on pages like this: http://en.wikipedia.beta.wmflabs.org/wiki/Earthworm https://www.mediawiki.org/wiki/Lightbox_demo http://en.wikipedia.org/wiki/Earthworm

[QA] Problem with Selenium, MobileFrontend, or VisualEditor?

2014-09-08 Thread Chris McMahon
I am hoping for some more eyeballs on this test failure:

Re: [QA] Problem with Selenium, MobileFrontend, or VisualEditor?

2014-09-08 Thread Chris McMahon
Just BTW, first known failure of this kind was 28 August. -C On Mon, Sep 8, 2014 at 10:17 AM, Željko Filipin zfili...@wikimedia.org wrote: On Mon, Sep 8, 2014 at 5:36 PM, Chris McMahon cmcma...@wikimedia.org wrote: What's happening is that Selenium is trying to enter a string

Re: [QA] IE11 support for Selenium/Webdriver

2014-09-06 Thread Chris McMahon
On Sat, Sep 6, 2014 at 5:59 AM, Željko Filipin zfili...@wikimedia.org wrote: On Fri, Sep 5, 2014 at 4:47 PM, Chris McMahon cmcma...@wikimedia.org wrote: Right now IE11 support is blocked by this issue: https://code.google.com/p/selenium/issues/detail?id=6511 But wait, how come Sauce Labs

[QA] IE11 support for Selenium/Webdriver

2014-09-05 Thread Chris McMahon
So the worldwide Selenium Conference is going on right now in Bangalore, and I saw just now that Jim Evans announced that Microsoft has committed to making Webdriver work for desktop IE11. Right now IE11 support is blocked by this issue: https://code.google.com/p/selenium/issues/detail?id=6511

Re: [QA] Browser test are now throttled

2014-08-25 Thread Chris McMahon
On Fri, Aug 22, 2014 at 12:19 PM, Antoine Musso hashar+...@free.fr wrote: Hello, The browser tests running on Jenkins are often failing due to some mysterious errors (timeout, network access, machine overloaded, race condition or whatever). It's not all that mysterious: most of these

[QA] Upcoming work on test environments

2014-08-19 Thread Chris McMahon
The thread about beta labs issues points up some issues that we in QA have been dealing with for some time, but that only affects individual teams from time to time mostly. The Release Engineering group is wrapping up one quarter's work and planning the next one, so this is a good time to think

Re: [QA] Flagging some Beta Labs issues

2014-08-19 Thread Chris McMahon
On Tue, Aug 19, 2014 at 3:09 PM, Jon Robson jrob...@wikimedia.org wrote: FWIW I think the more people with different use cases using beta labs the better I would like to keep us heavily using beta labs to improve the quality of the software we push out. Jon makes a nice point. The

Re: [QA] Flagging some Beta Labs issues

2014-08-18 Thread Chris McMahon
On Mon, Aug 18, 2014 at 4:35 PM, Maryana Pinchuk mpinc...@wikimedia.org wrote: Greetings, QAers, I'm not entirely sure who the point person for Beta Labs is currently, but I wanted to make sure you guys are aware that there have been a lot of issues (i.e., partial or total outages) with this

Re: [QA] Help setting up Git

2014-07-30 Thread Chris McMahon
You'll need to read and follow the steps described here: http://www.mediawiki.org/wiki/Gerrit/Tutorial -Chris On Wed, Jul 30, 2014 at 10:48 AM, Muzammil Rajwadkar muzzy...@hotmail.com wrote: Hey guys I am continuing with my Git setup however, I ran into a problem. I have navigated to my

Re: [QA] Pair programming for fun and profit

2014-07-30 Thread Chris McMahon
Hi Ritu, I understand that you visit the WMF office in San Francisco from time to time? Thanks for volunteering! -Chris On Wed, Jul 30, 2014 at 2:09 PM, Ritu Swain swain.r...@gmail.com wrote: Hi Željko, My name is Ritu Swain and I joined Wikimedia Foundation last month as a volunteer QA

Re: [QA] Two big improvements for flaky tests

2014-07-25 Thread Chris McMahon
On Fri, Jul 25, 2014 at 10:49 AM, Arthur Richards aricha...@wikimedia.org wrote: Wonderful news, thank you Chris :) Is https://bugzilla.wikimedia.org/show_bug.cgi?id=68465 related to the db issues that are now resolved, or is that a different issue? I believe that the readonly db is the

[QA] no HTTPS on beta labs

2014-07-22 Thread Chris McMahon
Nothing is listening at https://en.wikipedia.beta.wmflabs.org/ HTTPS ___ QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa

Re: [QA] no HTTPS on beta labs

2014-07-22 Thread Chris McMahon
Nothing is listening at https://en.wikipedia.beta.wmflabs.org/ HTTPS used to respond, but it would provide an invalid certificate, forcing the user to confirm the navigation. Now HTTPS seems to be gone altogether. I'm not sure if this was deliberate or not, or even if it should be changed, I

[QA] new shared method protect available for browser tests

2014-07-18 Thread Chris McMahon
Example: on(APIPage).protect Title of page to be protected, Reason for protection sets the level of protection to edit=sysop|move=sysop We have needed this in particular for a MobileFrontend test that was jumping through some huge hoops to protect a page. It is now universally available in

[QA] Status: VE, MF, Flow, and gallium

2014-07-17 Thread Chris McMahon
MobileFrontend build is green, VisualEditor build is green. Flow build is almost green. Right now the biggest problem we have is performance on the Jenkins host (gallium I presume). What we are seeing is a number of tests in the automatic build runs failing with the message getaddrinfo: Name or

Re: [QA] Testing thanks: How to do a browser test which involves 2 users

2014-07-17 Thread Chris McMahon
On Thu, Jul 17, 2014 at 3:08 PM, Chris Steipp cste...@wikimedia.org wrote: When we use central auth to login to different wikis, I've been setting another environment variables for each I like this. My only request is that you please do not explicitly logout Selenium_user, as that can affect

Re: [QA] Failing MobileFrontend browser tests

2014-07-16 Thread Chris McMahon
On Wed, Jul 16, 2014 at 2:37 PM, S Page sp...@wikimedia.org wrote: So I filed *Bug 68125* https://bugzilla.wikimedia.org/show_bug.cgi?id=68125 - browser tests failing with getaddrinfo: Name or service not known (SocketError) This seems to be in every case some test not sending any command to

Re: [QA] [WikimediaMobile] Failing MobileFrontend browser tests

2014-07-14 Thread Chris McMahon
On Mon, Jul 14, 2014 at 5:52 AM, Željko Filipin zfili...@wikimedia.org wrote: On Fri, Jul 11, 2014 at 9:35 PM, Arthur Richards aricha...@wikimedia.org wrote: Chris McMahon sent an email announcing this on June 6 [0]. In addition, Chris said 'The tests for MF on beta labs running in headless

Re: [QA] Reg::Timeout error on executing links.feature

2014-07-11 Thread Chris McMahon
The links.feature test is unique in that the browser running the test must have focus in order for the clickable link text generated from the link inspector to appear. When running this locally, if you start the test and then use some other browser window or application while the test is running,

Re: [QA] browser tests race condition with login/logout

2014-07-01 Thread Chris McMahon
, Antoine Musso hashar+...@free.fr wrote: Hello, We have a bunch of browser tests that: - potentially runs concurrently - target the same wiki - use the same username During our weekly checkin, Chris McMahon raised the issue of tests being suddenly logged out which obviously break tests badly

[QA] future of gadget tests from qa/browsertest repo

2014-06-30 Thread Chris McMahon
We have two tests in the /qa/browsertests repo, and Matt wants to know what is going to happen to them: https://gerrit.wikimedia.org/r/#/c/142359/ Tests for gadget certainly don't belong in /mediawiki/core. Since gadgets are not in gerrit (or under source control at all) we can't assign them to

Re: [QA] Moving builds off Cloudbees: VisualEditor, MobileFrontend, Flow

2014-06-30 Thread Chris McMahon
There are at least two software issues to be rectified, not sure if it's the code or the tests. Juliusz should be looking at those today. On Mon, Jun 30, 2014 at 12:52 PM, Jon Robson jrob...@wikimedia.org wrote: The Chrome [1] and Firefox [2] builds are still yet to pass so no. I've not

Re: [QA] Moving builds off Cloudbees: VisualEditor, MobileFrontend, Flow

2014-06-30 Thread Chris McMahon
-chrome-sauce/ I'm not really sure what is actually failing in these tests... Are there any bugs open or any kind of indication when we can expect these problems to be fixed? On Mon, Jun 30, 2014 at 12:58 PM, Chris McMahon cmcma...@wikimedia.org wrote: There are at least two software issues

Re: [QA] RVM and/or rbenv

2014-06-28 Thread Chris McMahon
On Sat, Jun 28, 2014 at 4:33 AM, Željko Filipin zfili...@wikimedia.org wrote: This is mostly question for Dan Duvall, but I wanted to ask it in public, so others could contribute too. Dan, what are you thoughts about RVM[1] and rbenv[2]? Should we use one or the other exclusively? Or are

[QA] Moving builds off Cloudbees: VisualEditor, MobileFrontend, Flow

2014-06-16 Thread Chris McMahon
Hi all, After some doing some spikes last week, Željko and I moved our first six official builds from Cloudbees to WMF Jenkins. They all run against beta labs and they all use our nifty WMF SauceLabs account: VisualEditor (currently failing with a known problem in beta labs):

[QA] Presenting at Wikimania!

2014-06-13 Thread Chris McMahon
I just confirmed that the proposal for the talk Finding and fixing software bugs for the Wikipedias [1] got accepted. Unless something changes, I'll present it at 10:00 on Saturday, 9 August in London. [1]

[QA] Now's a good time to try the Android app!

2014-06-10 Thread Chris McMahon
This went out to the WMF staff-only mail list just now, but I think it's appropriate for QA (slightly edited... :-) ) -- Forwarded message -- Hi folks, We're ~2 weeks from release of the new Android (native) Wikipedia app. Now's a good time for *everyone* with an Android device

[QA] VE build run on SauceLabs via WMF Jenkins

2014-06-09 Thread Chris McMahon
Željko and I (mostly Željko!) got this running today: https://saucelabs.com/tests/e580fcbe2d5745d59f37600c6e0e9015 Still more config and refactoring work to do in Jenkins Job Builder, but this is pretty great. It has been a long road to get here at all. -Chris

Re: [QA] [EE] migrating Flow tests to WMF Jenkins

2014-06-09 Thread Chris McMahon
On Fri, Jun 6, 2014 at 6:20 PM, James Forrester jforres...@wikimedia.org wrote: On 6 June 2014 16:21, Chris McMahon cmcma...@wikimedia.org wrote: On Fri, Jun 6, 2014 at 4:16 PM, Matthew Flaschen mflasc...@wikimedia.org wrote: Are they voting or non-voting? I think browser tests should

Re: [QA] [EE] migrating Flow tests to WMF Jenkins

2014-06-09 Thread Chris McMahon
On Mon, Jun 9, 2014 at 10:49 AM, James Forrester jforres...@wikimedia.org wrote: On 9 June 2014 10:46, Chris McMahon cmcma...@wikimedia.org wrote: I encourage thinking along the lines of if it's a bug in beta labs then it is a bug, period, and we need to fix it. ​I would be rather stronger

[QA] new QA area: Vagrant and Ruby

2014-06-06 Thread Chris McMahon
Dan Duvall, the new Automation Engineer at WMF has started working on updating and awesomeifying our existing Vagrant configuration. If you're interested in some deep Ruby and virtual machines, feel free to have a look: https://gerrit.wikimedia.org/r/#/c/137483/ -Chris

[QA] migrating Flow tests to WMF Jenkins

2014-06-06 Thread Chris McMahon
Hello Flow folk, Just a quick note to say that now that Zeljko is back from parental leave, we are proceeding to dismantle the Cloudbees Jenkins jobs that run browser tests (including for Flow)[1] in favor of jobs on WMF Jenkins. [2] The tests for Flow on beta labs running in headless Firefox

[QA] migrating MobileFrontend browser tests to WMF Jenkins

2014-06-06 Thread Chris McMahon
Hello MobileFrontenders, Just a quick note to say that now that Zeljko is back from parental leave, we are proceeding to dismantle the Cloudbees Jenkins jobs that run browser tests (including for MobileFrontend)[1] in favor of jobs on WMF Jenkins. [2] The tests for MF on beta labs running in

Re: [QA] When are we moving away from Jenkins?

2014-06-05 Thread Chris McMahon
Today Željko and I have some builds working in WMF Jenkins, for MobileFrontend and for VisualEditor on headless Firefox. The MF build is working fine except for one mysterious failure: pageactions_protected.feature:9 doesn't see a toast div and it should. Note that the step Then I see a toast

Re: [QA] When are we moving away from Jenkins?

2014-06-05 Thread Chris McMahon
On Thu, Jun 5, 2014 at 1:48 PM, Jon Robson jrob...@wikimedia.org wrote: Is there any way to see a screencast of what happens here? Not yet. I suspect this is because the user that runs the Selenium tests does not have the right user permissions According to the README.mediawiki file

Re: [QA] When are we moving away from Jenkins?

2014-06-05 Thread Chris McMahon
On Thu, Jun 5, 2014 at 3:28 PM, Jon Robson jrob...@wikimedia.org wrote: Mmm. It looks like everything is working correctly. Is it guaranteed that the user is anonymous at the start of the test? Should be. We'll get some tracing on it tomorrow morning.

Re: [QA] Repeating failing tests - any ideas?

2014-06-02 Thread Chris McMahon
On Mon, Jun 2, 2014 at 1:43 PM, Jon Robson jdlrob...@gmail.com wrote: There is a consistent failure on 2 tests in the Chrome build. Firefox build not effected. Given(/^I am viewing the site in tablet mode$/) do @browser.window.resize_to(768, 1024) end Watching these run, and checking

Re: [QA] Test failures - Trello: Introducing Sellello

2014-05-22 Thread Chris McMahon
On Thu, May 22, 2014 at 9:08 AM, Juliusz Gonera jgon...@wikimedia.orgwrote: On 05/22/2014 02:57 PM, Jon Robson wrote: When a browser test has been failing for a considerable time e.g. for the last 3 builds it will post a card to Trello. Once a long long time ago(*) I wrote a test harness

Re: [QA] Reg::Running of links.feature tests

2014-05-20 Thread Chris McMahon
Hi Jagori, On Mon, May 19, 2014 at 5:36 PM, jagori samajdar jagor...@gmail.com wrote: - at should I set for MEDIAWIKI_API_URL?It's prompting me to specify this env variable before running. Generally the api URL is yourwiki/w/api.php . For example export

[QA] browser acceptance/smoke tests for mediawiki new install, upgrade, arbitrary language

2014-05-20 Thread Chris McMahon
Hi Mark and Chris and everyone on the QA list, Some time ago we discussed creating a set of browser-based tests to validate the state of a newly installed or upgraded mediawiki wiki. Last week I created one of those: https://gerrit.wikimedia.org/r/#/c/133507/ Now I would like to get some review

[QA] Fwd: browser acceptance/smoke tests for mediawiki new install, upgrade, arbitrary language

2014-05-20 Thread Chris McMahon
On Tue, May 20, 2014 at 2:10 PM, Chris Steipp cste...@wikimedia.org wrote: I think that's a great start. Eventually, I'd like to have tests for all of the major features of a default wiki (logs and recent changes, page moves and deletes, account creation, user login, watchlist handling,

Re: [QA] all browser tests failing locally for me

2014-05-20 Thread Chris McMahon
The target wiki should be controlled by the local environment variable MEDIAWIKI_URL e.g. $ export MEDIAWIKI_URL=http://en.m.wikipedia.beta.wmflabs.org/wiki/ On Tue, May 20, 2014 at 5:34 PM, Arthur Richards aricha...@wikimedia.orgwrote: +qa On May 20, 2014 5:28 PM, Ryan Kaldari

Re: [QA] all browser tests failing locally for me

2014-05-20 Thread Chris McMahon
no idea how it was always testing against the mobile version before, but it used to work fine. Ryan On Tue, May 20, 2014 at 5:46 PM, Chris McMahon christopher.mcma...@gmail.com wrote: The target wiki should be controlled by the local environment variable MEDIAWIKI_URL e.g. $ export

Re: [QA] Reg::Page Objects

2014-05-19 Thread Chris McMahon
Hi Jagori, We discussed this a little last week, and now that you've written this report, I remember why we made the Links test work this way. On Mon, May 19, 2014 at 6:31 AM, jagori samajdar jagor...@gmail.com wrote: Certain elements like span which have been defined as below in

Re: [QA] Introduction - Exploratory Tester - Contribution

2014-05-19 Thread Chris McMahon
Hi Jagan, On Sun, May 18, 2014 at 10:07 AM, jagannath sriramulu jagannat...@hotmail.com wrote: Hi all, I would like to contribute towards testing activities for Wikimedia. Currently I am open about my participation(areas of participation) and can spend up to 4 hours per day. Thanks!

[QA] VisualEditor testability status right now

2014-05-19 Thread Chris McMahon
Hi QAers, I know some of you are working with VisualEditor. I wanted to let you know that VE is significantly broken right now on both beta labs and also on test2wiki and mediawiki.org. The issues are mostly for Firefox, but you may also encounter strange behavior from VE in Chrome. We are

Re: [QA] VisualEditor testability status right now

2014-05-19 Thread Chris McMahon
I'm still not calling it until I actually see it. I'm still seeing issues with the Links interface. I've been talking to Rummana about those. On Mon, May 19, 2014 at 10:40 AM, Greg Grossmeier g...@wikimedia.orgwrote: quote name=Chris McMahon date=2014-05-19 time=10:23:04 -0700 Hi QAers

Re: [QA] VisualEditor testability status right now

2014-05-19 Thread Chris McMahon
VE should be mostly back to normal. I am aware of one small issue in the Links inspector, and if you find any issues, please speak up, we just hacked in a really big change to the UI code. -Chris On Mon, May 19, 2014 at 10:56 AM, Chris McMahon christopher.mcma...@gmail.com wrote: I'm still

Re: [QA] Chrome test has been failing for some time

2014-05-13 Thread Chris McMahon
On Tue, May 13, 2014 at 9:25 AM, Jon Robson jrob...@wikimedia.org wrote: Chris, this test has been failing for some time and I can't quite work out why. In Chrome, it looks like uploaded_image_link always points to the upload link at the top of the page when it is loaded. I suspect Chrome

Re: [QA] Chrome test has been failing for some time

2014-05-13 Thread Chris McMahon
For the sake of completeness, here's why this test is failing: The regular UploadWizard checks for duplicate images on the target wiki and prevents users from uploading if it finds a duplicate image. (This btw is why the regular UploadWizard test uses the ChunkyPNG utility to create near-unique

[QA] Fwd: Chrome test has been failing for some time

2014-05-13 Thread Chris McMahon
On Tue, May 13, 2014 at 1:22 PM, Jon Robson jrob...@wikimedia.org wrote: Thanks for documenting this Chris! I'm not sure if this should be a bug - maybe uploads to Special:Uploads of this form should show an error message saying We already have this photo. ? I am tempted to leave it the

[QA] welcome Noah Sussman

2014-05-08 Thread Chris McMahon
I've been encouraging Noah to use some of our test environments and code repos for his own projects in recent times. He was part of the team at Etsy that created their Continuous Delivery systems, and he knows a thing or two about Selenium too. Nice to see you here, Noah... -Chris

[QA] Browsertests using only WMF Jenkins and beta labs!

2014-04-18 Thread Chris McMahon
Hello QAers, Thanks to Antoine and Željko, we are now running a number of builds for browsertests that use the WMF Jenkins instance with headless Firefox to run against the beta labs test environment directly. https://integration.wikimedia.org/ci/view/BrowserTests/ This is really exciting. It

Re: [QA] QA Code review help needed

2014-04-15 Thread Chris McMahon
Chris checked it out it wasn't based properly of the latest version of master. Help still needed on the other patchsets! I'm being very attentive today! :) On Tue, Apr 15, 2014 at 10:09 AM, Chris McMahon cmcma...@wikimedia.org wrote: Hi Jon, This one https://gerrit.wikimedia.org/r

[QA] documentation for cross-repo browser test features

2014-04-11 Thread Chris McMahon
https://www.mediawiki.org/wiki/Browser_testing/shared_features We have implemented a number of features in the mediawiki_selenium Ruby gem that are available to any repo that would like to use them. I thought it would be useful to document what we have so far. Comments and criticism always

Re: [QA] My first browser tests in VisualEditor

2014-03-28 Thread Chris McMahon
Thanks Vikas! You asked some really great, insightful questions along the way, and the pastebin examples you showed me were very good and right to the point. And it's a good test too. :-) -Chris On Fri, Mar 28, 2014 at 1:14 AM, Vikas Yaligar vikasyaligar...@gmail.comwrote: Hello, My

Re: [QA] QA Tests: Waiting for things to disappear

2014-03-27 Thread Chris McMahon
Ha! I was thinking of this (and I continue to think of this) as two separate problems, but I do find myself using the construct in 121543https://gerrit.wikimedia.org/r/121543 a lot. However, I am still concerned about the token/cookie issue and I'm still trying to get to the bottom of why it

Re: [QA] Run UploadWizard API integration test against production...

2014-03-26 Thread Chris McMahon
On Wed, Mar 26, 2014 at 5:31 AM, Željko Filipin zfili...@wikimedia.orgwrote: On Thu, Jan 30, 2014 at 6:27 PM, Aaron Arcos aarcos.w...@gmail.comwrote: I again Željko, I just checked the consolehttps://wmf.ci.cloudbees.com/view/r-uw/job/UploadWizard-commons.wikimedia.org/14/consoleof the

[QA] I proposed a QA talk for Wikimania in London

2014-03-26 Thread Chris McMahon
Comments welcome! https://wikimania2014.wikimedia.org/wiki/Submissions/Finding_and_fixing_software_bugs_for_the_Wikipedias -Chris PS thanks Greg for reading it over. ___ QA mailing list QA@lists.wikimedia.org

[QA] how often to run browser tests?

2014-03-25 Thread Chris McMahon
There has been some confusion and lack of consensus about this for some time. I'd like to discuss the issue of frequency so that we at least have some knowledge, even if we might disagree on the details. The idea of running tests for every commit is for unit tests only. And by unit tests I mean

Re: [QA] Emails and timeout errors

2014-03-25 Thread Chris McMahon
On Tue, Mar 25, 2014 at 3:34 PM, Jon Robson jdlrob...@gmail.com wrote: I'm seeing lots of tests failing with too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) a Long term view: How do we stop these? We port our Jenkins builds for browser tests from Cloudbees to WMF

Re: [QA] Page Object design pattern: where to put element identifiers/selectors

2014-03-21 Thread Chris McMahon
On Fri, Mar 21, 2014 at 2:56 AM, Željko Filipin zfili...@wikimedia.orgwrote: On Thu, Mar 20, 2014 at 11:54 PM, Chris McMahon cmcma...@wikimedia.orgwrote: Adhere to the Page Object design pattern: keep your page element identifiers in the /support/pages/*page.rb files ONLY

Re: [QA] MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox - Build # 429 - Still Failing!

2014-03-20 Thread Chris McMahon
This is all the stuff we fixed this week. -C On Thu, Mar 20, 2014 at 8:30 AM, Željko Filipin zfili...@wikimedia.orgwrote: On Tue, Mar 18, 2014 at 6:15 PM, Jon Robson jrob...@wikimedia.org wrote: These tests shouldn't be running on beta labs since the @custom-browser tag doesn't seem to

[QA] Page Object design pattern: where to put element identifiers/selectors

2014-03-20 Thread Chris McMahon
The Page Object design pattern divides browser tests into three areas: the controller, where the test is created (this is a description of the test): the steps of the test, which are actions taken by the test (the steps are verbs, actions, navigation from one place to another and checking on

Re: [QA] [WikimediaMobile] Problems with QA MobileFrontend Chrome builds

2014-03-18 Thread Chris McMahon
- is this documented anywhere? https://bugzilla.wikimedia.org/show_bug.cgi?id=62614 https://bugzilla.wikimedia.org/show_bug.cgi?id=62583 On Thu, Mar 13, 2014 at 2:16 PM, Chris McMahon christopher.mcma...@gmail.com wrote: On Thu, Mar 13, 2014 at 3:06 PM, Arthur Richards aricha

Re: [QA] [WikimediaMobile] Problems with QA MobileFrontend Chrome builds

2014-03-18 Thread Chris McMahon
On Tue, Mar 18, 2014 at 10:50 AM, Chris McMahon christopher.mcma...@gmail.com wrote: https://code.google.com/p/selenium/wiki/ChromeDriver with ChromeDriver in a directory in your path (probably /user/bin/chromedriver) just set your env var BROWSER_LABEL=chrome and it should Just Work

Re: [QA] [WikimediaMobile] Problems with QA MobileFrontend Chrome builds

2014-03-18 Thread Chris McMahon
this be documented somewhere? On Tue, Mar 18, 2014 at 11:13 AM, Chris McMahon cmcma...@wikimedia.org wrote: You shouldn't have to start Chromedriver, the framework will do that for you. * Chrome is installed in the normal location * chromedriver is downloaded and exists in /usr/bin

  1   2   3   >