[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. Note that the change quoted above was premature and it is not true that all Wikibase browser tests have been migrated to nodejs. We still have quite some work there. We intend to restore said daily job soon, but let's track this separately, as the status of thi

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I see. Selenium view lists all jobs that have `selenium` in name (actually `^(?=.*selenium)((?!noselenium).)*$`). The only Wikibase related job is `selenium-daily-beta-Wikibase` and it's green. I guess you can resolved this task then. 😁 TASK DETAIL http

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Addshore added a comment. Yes, but this ticket is talking about daily browser tests, not on patch browser tests. The link in the description in the TODO section for test is https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFOR

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#5317884 , @Addshore wrote: > I can't find the jobs for test right now? Are these even running any more? Or are we just on beta now? I'm not sure what you mean. This? Random patch: http

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-04-02T23:30:26Z] Synchronized wmf-config/InitialiseSettings.php: SWAT Add new WMCS IP range to wgRateLimitsExcludedIps T167432 (duration: 00m 57s) TASK DETAIL https://phab

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 482640 **merged** by jenkins-bot: [operations/mediawiki-config@master] Add new WMCS IP range to $wgRateLimitsExcludedIps https://gerrit.wikimedia.org/r/482640 TASK DETAIL https://phabricator.wikimedia.org/T167432 EMAIL PREFERENCES https://phabrica

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 482640 had a related patch set uploaded (by Zfilipin; owner: Hashar): [operations/mediawiki-config@master] Add new WMCS IP range to $wgRateLimitsExcludedIps https://gerrit.wikimedia.org/r/482640 TASK DETAIL https://phabricator.wikimedia.org/T167432

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 482640 had a related patch set uploaded (by Hashar; owner: Hashar): [operations/mediawiki-config@master] Add new WMCS IP range to $wgRateLimitsExcludedIps https://gerrit.wikimedia.org/r/482640TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShtt

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 437486 abandoned by Zfilipin: WIP Increase timeout of daily Selenium jobs Reason: selenium-Wikibase-chrome fails even with 5 hour timeout https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/buildTimeTrend https://gerrit.wikimedia.org/r/437486TASK DE

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. selenium-Wikibase-chrome is updated and selenium-Wikibase-chrome/MEDIAWIKI_ENVIRONMENT=test is running. 🏃TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Lydia_P

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 437486 had a related patch set uploaded (by Zfilipin; owner: Zfilipin): [integration/config@master] WIP Increase timeout of daily Selenium jobs https://gerrit.wikimedia.org/r/437486TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabri

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. selenium-Wikibase-chrome/MEDIAWIKI_ENVIRONMENT=test is timing out even after 5 hours. I will increase the timeout to 10 hours just to see how long it takes. Once it passes, I will reduce it to a reasonable number of hours. 😅TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I am not sure what happened, selenium-Wikibase-chrome/test now takes 20-30 minutes instead of 4-5 hours. 🤔TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Lydia_

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. 434908 is merged and deployed, I have started the job. We will know more in 5 hours. 😅TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Lydia_Pintscher, Stashbot,

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 434908 merged by jenkins-bot: [integration/config@master] Increase timeout of daily Selenium jobs https://gerrit.wikimedia.org/r/434908TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 434908 had a related patch set uploaded (by Zfilipin; owner: Zfilipin): [integration/config@master] Increase timeout of daily Selenium jobs https://gerrit.wikimedia.org/r/434908TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricato

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Looks like even 4 hours 🕰 was not enough. Please create a patch that increases the timeout to 5 hours. ⏳TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Lydia_Pi

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 434037 merged by jenkins-bot: [integration/config@master] Temporarily increase the timeout of selenium jobs https://gerrit.wikimedia.org/r/434037TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/e

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. @zeljkofilipin: actually I went ahead at the hackathon and made https://gerrit.wikimedia.org/r/#/c/434037/.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: WMDE-leszekCc: Lydia_Pin

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 434037 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek): [integration/config@master] Temporarily rise up the tiemout of selenium jobs https://gerrit.wikimedia.org/r/434037TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShtt

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. @zeljkofilipin: as discussed today, if you could rise the timeout limit for the job targeting test to say 240 minutes, it would hopefully be enough for the job to finish. We could then see if tests are passing after the Cirrus-related adjustments I've been making weeks

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 409918 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Wait for Cirrus index update in browser tests when needed https://gerrit.wikimedia.org/r/409918TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 409918 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek): [mediawiki/extensions/Wikibase@master] Wait for Cirrus index update in browser tests when needed https://gerrit.wikimedia.org/r/409918TASK DETAILhttps://phabricator.wikimedia.org/T167432

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 399622 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Try out if waiting a bit after creating property make search results appear as expected https://gerrit.wikimedia.org/r/399622TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCESh

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 399622 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek): [mediawiki/extensions/Wikibase@master] Try out if waiting a bit after creating property make search results appear as expected https://gerrit.wikimedia.org/r/399622TASK DETAILhttps://pha

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 390394 merged by jenkins-bot: [integration/config@master] Wikibase daily browser test does not use Sauce Labs https://gerrit.wikimedia.org/r/390394TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Fixing test failures when targeting testwiki?TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Lydia_Pintscher, Stashbot, Ladsgroup, hoo, Addshore, aude, gerritbo

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Testwiki job now looks much better. F10789456: Screen Shot 2017-11-14 at 13.08.15.png 73 (out of 74) failures are Watir::Wait::TimeoutError After a quick look at the screenshots, looks like the problem is in No match was found error message. F10789825: Adding refe

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I do not know what went wrong 😕 I am re-running the job.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Stashbot, Ladsgroup, hoo, Addshore, aude, gerritbot, Pok

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. hmm, quite a number of odd failures, e.g. unknown error: Chrome failed to start: exited abnormally browser window was closed (Watir::Exception::NoMatchingWindowFoundException) Net::ReadTimeout (Net::ReadTimeout) Should I assume this was some hiccup and re-run th

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. @Ladsgroup has deployed the config change. The job is running: selenium-Wikibase-chrome/test/9.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Stashbot, Ladsgro

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2017-11-13T14:10:24Z] Synchronized wmf-config/InitialiseSettings.php: Whitelist jenkins in test wiki (T167432) (duration: 00m 47s)TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 390975 merged by jenkins-bot: [operations/mediawiki-config@master] Whitelist jenkins in test wiki https://gerrit.wikimedia.org/r/390975TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 390975 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani): [operations/mediawiki-config@master] Whitelist jenkins in test wiki https://gerrit.wikimedia.org/r/390975TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://ph

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 384500 abandoned by Zfilipin: WIP Run Wikibase daily browser tests on Jenkins Reason: just an experiment https://gerrit.wikimedia.org/r/384500TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/ema

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. Thanks @zeljkofilipin! This is really looking good. We'll have a Jenkins IPs whitelisted and see what actually failures are actually there on test. This is most likely going to happen on Monday.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps:/

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. $ diff output0/selenium-Wikibase output3/selenium-Wikibase-chrome 13c13 < BrowserTests --- > DebianJessie && contintLabsSlave 42c42 < BrowserTests --- > DebianJessie && contintLabsSlave 145a146,156 > # screenshots > export SCREENSHOT_FAILURES=true

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have created selenium-Wikibase-chrome (config).TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: Ladsgroup, hoo, Addshore, aude, gerritbot, PokestarFan, Tobi_WM

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 390394 had a related patch set uploaded (by Zfilipin; owner: Zfilipin): [integration/config@master] WIP Wikibase daily browser test does not use Sauce Labs https://gerrit.wikimedia.org/r/390394TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCESht

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Ladsgroup added a comment. I can do the whitelistingTASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, LadsgroupCc: Ladsgroup, hoo, Addshore, aude, gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thi

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. Thanks @zeljkofilipin! I will try to look at those failures in more detail when I am on a stable network connection. @Ladsgroup: Regarding whitelisting IPs of Jenkins, is this something you might be able to look into, or should I ask further?TASK DETAILhttps://phabrica

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. It looks to me that Jenkins' IPs have to be whitelisted for testwiki, like it was done for beta cluster. That should fix The save has failed. (failed-save) (MediawikiApi::ApiError).TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricato

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I took a look at selenium-Wikibase-T167432/test failures: 1 RSpec::Expectations::ExpectationNotMetError 26 The save has failed. (failed-save) (MediawikiApi::ApiError) 56 Watir::Wait::TimeoutError TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCESh

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I took a look at a few selenium-Wikibase/test failures (example: jenkins, sauce labs) and all of them failed with error No match was found. F10691995: 0012screenshot.pngTASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. So far I was focused on getting the job running for beta. I did not even look at the test job. For beta, the original job (selenium-Wikibase) has 1 failure. The new job (selenium-Wikibase-T167432) has no failures. selenium-Wikibase/beta: 212 scenarios (1 failed, 21

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. oh, wait. Looking a screen caps it looks like those failures of the test job happen far before even starting to test anything. Is it intentional?TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpr

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. So it seems @zeljkofilipin that the beta job run without failures but there are 95 failures on test job. I guess still have plenty of stuff to look into then?TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 388062 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Wait for $.cookie to be loaded in browser tests https://gerrit.wikimedia.org/r/388062TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/setting

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#3729386, @WMDE-leszek wrote: I've just run another run of the job and there are 4 failures. So it looks like I only need to fix those, and we could then go on with the new jobs instead of the one running tests on sauce labs? Correct. The only remaining t

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 388062 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek): [mediawiki/extensions/Wikibase@master] Wait for $.cookie to be loaded in browser tests https://gerrit.wikimedia.org/r/388062TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREF

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. Okay, thanks. That's good news! I've just run another run of the job and there are 4 failures. So it looks like I only need to fix those, and we could then go on with the new jobs instead of the one running tests on sauce labs?TASK DETAILhttps://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#3726174, @WMDE-leszek wrote: @zeljkofilipin: I don't have rights to see the config of the job, so I am not sure if the user factory is now enabled or not? It is. The config change uses master branch. Looks like most failures of that last run of yours we

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 386416 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Do not fail on browser tests when browser errors are expected to happen https://gerrit.wikimedia.org/r/386416TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricat

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 386416 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek): [mediawiki/extensions/Wikibase@master] Do not fail on browser tests when browser errors are expected to happen https://gerrit.wikimedia.org/r/386416TASK DETAILhttps://phabricator.wikimed

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. @zeljkofilipin: I don't have rights to see the config of the job, so I am not sure if the user factory is now enabled or not? Looks like most failures of that last run of yours were because of not finding items/properties. This seems odd, and also does appear to me loc

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. 01:13:27.065 212 scenarios (84 failed, 128 passed) 01:13:27.066 2553 steps (84 failed, 564 skipped, 1905 passed) 01:13:27.066 73m22.232sTASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferenc

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#3709235, @WMDE-leszek wrote: @zeljkofilipin just as heads up: @Ladsgroup was so kind to whitelist instances where these jobs run, so there is no problem with the edit rate limit. With this change there still are 6 failures, but looks like this really time

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 386343 merged by jenkins-bot: [operations/mediawiki-config@master] labs: Whitelist jenkins and Sauce labs IP ranges https://gerrit.wikimedia.org/r/386343TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 386343 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani): [operations/mediawiki-config@master] labs: Whitelist jenkins and Sauce labs IP ranges https://gerrit.wikimedia.org/r/386343TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PRE

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Videos are here: https://integration.wikimedia.org/ci/view/Selenium/job/selenium-Wikibase-T167432/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/lastCompletedBuild/ https://integration.wikimedia.org/ci/view/Seleni

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Screenshots are there: https://integration.wikimedia.org/ci/job/selenium-Wikibase-T167432/21/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/ https://integration.wikimedia.org/ci/job/selenium-Wikibase-T167432/21/BR

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. @Tobi_WMDE_SW It should already take screenshots, but I have just checked and I do not see them anywhere. :| I will check what is wrong with screenshots and how to enable video recording.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phab

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. @zeljkofilipin Thanks! I have a related question, would it be possible to configure the job in a way so it takes screenshots/videos for the failing scenarios just like the browsertests that run on every commit do it?TASK DETAILhttps://phabricator.wikimedia.org/T167432E

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

WMDE-leszek added a comment. Thanks a lot @zeljkofilipin. I believe we should take it over from here.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, WMDE-leszekCc: hoo, Addshore, aude, gerritbo

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. This is no longer CI/Ruby problem, but beta/configuration problem, and I do not have experience with that. I am not sure what to do next.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferen

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Hm, seems it works if I use Selenium_user as both users. Testing.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hoo, Addshore, aude, gerritbot, PokestarFan, To

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have updated permissions for Selenium_user2 to be the same as Selenium_user, but it did not help, I am still getting permissiondenied. :|TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. We should definitely use the user factory and somehow whitelist Jenkin's IP address or disable AbuseFilter.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Tobi_WMD

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have submitter patch 384500 that logs the user in. The patch also disables user_factory, since I was getting this error message: MediawikiApi::CreateAccountError: Visitors to this wiki using your IP address have created 6 accounts in the last day, which is the maxi

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

gerritbot added a comment. Change 384500 had a related patch set uploaded (by Zfilipin; owner: Zfilipin): [mediawiki/extensions/Wikibase@master] WIP Run Wikibase daily browser tests on Jenkins https://gerrit.wikimedia.org/r/384500TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCE

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. The error message is actionthrottledtext.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thiemowmde, WMDE-leszek, z

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. The screenshot says: Could not save due to an error. The save has failed. As an anti-abuse measure, you are limited from performing this action too many times in a short space of time, and you have exceeded this limit. Please try again in a few minutes. TASK DETA

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. F10257452: failed-save.pngTASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thiemowmde, WMDE-leszek, zeljkofilipin, h

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. You could try to output the whole error object in the api-gem. I guess there's probably more detailed information in there than what is shown right now.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I am able to reproduce the problem from my machine, targeting beta cluster. $ MEDIAWIKI_ENVIRONMENT=beta bundle exec cucumber features/aliases.feature # Wikidata UI tests # # Author:: Tobias Gritschacher (tobias.gritschac...@wikimedia.de) # License:: GNU GPL v2+ # #

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. For creating properties the user is logged in, see the create_property method in property_page.rb. This is because only "propertycreators" can actually create properties on a Wikibase instance. For creating items this is not needed as also anons can create these. But I

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#3686814, @Tobi_WMDE_SW wrote: We are not logging in to create new items (as it is not needed and tests are speeded up), so probably the attempts to create items at such a high rate from an unknown IP are recognized as harmful. Could be that the IP from Sau

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have tried debugging locally, but I get Unrecognized value for parameter "action": wbeditentity. (unknown_action) (MediawikiApi::ApiError). cd mediawiki/vagrant vagrant up vagrant roles enable wikidata vagrant provision cd mediawiki/extensions/Wikibase/tests/bro

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. Another guess: We are not logging in to create new items (as it is not needed and tests are speeded up), so probably the attempts to create items at such a high rate from an unknown IP are recognized as harmful. Could be that the IP from SauceLabs was whitelisted..?TAS

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Tobi_WMDE_SW added a comment. The save has failed. (failed-save) (MediawikiApi::ApiError) /var/lib/gems/2.1.0/gems/mediawiki_api-0.6.0/lib/mediawiki_api/client.rb:209:in `send_request' /var/lib/gems/2.1.0/gems/mediawiki_api-0.6.0/lib/mediawiki_api/client.rb:230:in `raw_action' /var/lib/gems/2.1.0/g

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. To make it explicit: selenium-Wikibase-T167432 job runs fine when the browser is running at Sauce Labs when the browser is running at a local machine, a few tests randomly fail with failed-save error message. The only idea I have so far is that there is a timing p

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have switched to job back to sauce labs, and the failed-save failures are gone :| https://integration.wikimedia.org/ci/view/Selenium/job/selenium-Wikibase-T167432/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/15

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I was able to reproduce the problem on my machine, targeting beta cluster. I am not sure what is wrong. :|TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerrit

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Raw XML config files (easy to diff): selenium-Wikibase/config.xml selenium-Wikibase-T167432/config.xml TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritb

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. @Aleksey_WMDE @Tobi_WMDE_SW I am confused, why does selenium-Wikibase-T167432 fail so much with "wikibase-api-failed-save": "The save has failed."? As far as I can see, everything it the same as the original job (selenium-Wikibase), including usernames/passwords.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Looks like this is where the error message comes from: https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/repo/i18n/en.json#L329 "wikibase-api-failed-save": "The save has failed.",TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFEREN

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. 42 out of 69 failures (61%) are The save has failed. (failed-save) (MediawikiApi::ApiError)TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, PokestarFa

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. 212 scenarios (69 failed, 19 skipped, 124 passed)TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thiemowmde, WMDE-l

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. It is running, but failing a lot: https://integration.wikimedia.org/ci/job/selenium-Wikibase-T167432/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/9/consoleTASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

hashar added a comment. Sorry my bad. The saucelabs* Jenkins instances are for the daily jobs using saucelabs. They only have ruby installed. You would want the permanent jessie slaves that do come with Xvfb/chromium. Eg: node: DebianJessie && contintLabsSlaveTASK DETAILhttps://phabricator.wikime

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. Thanks, I have completely forgot about that. I have tried (from mw-set-env-mw-selenium.sh) # Selenium requires the chromedriver binary to be found in our PATH for path in /usr/lib/chromium-browser /usr/lib/chromium; do if test -d "$path"; then export PATH="$PATH:

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

hashar added a comment. Chromedriver is not in the default PATH. That is done by the slave script mw-set-env-mw-selenium bin/mw-set-env-mw-selenium.sh:for path in /usr/lib/chromium-browser /usr/lib/chromium; do Usually invoked via mw-selenium-setup.sh (which also append a found LocalSettings.php

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. In T167432#3649874, @hashar wrote: At first: keep the job tied to the BrowserTests label. it is exactly meant for that. Done, but it fails with Unable to find the chromedriver executable. BrowserTests assumes the browser is running at Sauce Labs, but that proved no

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

hashar added a comment. At first: keep the job tied to the BrowserTests label. it is exactly meant for that. Specially the Wikibase job takes a while and would keep a Nodepool instance busy for a while. Maybe that is enough to fix it? Then once it runs on BrowserTests, you would be able to repro

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I have recreated selenium-Wikibase-T167432 job following instuctions from T167432#3477666 and T167432#3478230. The main change is that instead of running on BrowserTests slaves, the job now runs on DebianJessie && contintLabsSlave. It fails with unknown error: Chro

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. This is what we have at the moment (from Selenium/Ruby/Selenium_Jenkins_job): F9877418: Selenium_Jenkins_job.png Sauce Labs proved not to be robust enough. Alternatives are to run everything on a Jenkins slave (from Selenium/Ruby/mwext-mw-selenium_Jenkins_job): F9

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. I am really busy with Selenium in Node.js lately. I will try to make some progress this week.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, Pokestar

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Aleksey_WMDE added a comment. Ok, thank you.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Aleksey_WMDECc: gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thiemowmde, WMDE-leszek, zeljkofilipin,

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

zeljkofilipin added a comment. @Aleksey_WMDE we are almost there, the job is passing. Unfortunately, I do not have the time to resolve it today, and I will be on vacation for a week. I will continue when I get back.TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabric

[Wikidata-bugs] [Maniphest] [Commented On] T167432: Run Wikibase daily browser tests on Jenkins

Aleksey_WMDE added a comment. Can I ask what the status is?TASK DETAILhttps://phabricator.wikimedia.org/T167432EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Aleksey_WMDECc: gerritbot, PokestarFan, Tobi_WMDE_SW, Jonas, thiemowmde, WMDE-leszek,

  1   2   >