[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2024-05-04 Thread Maintenance_bot 
Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, Maintenance_bot
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, Ullasoff, 
Danny_Benjafield_WMDE, S8321414, Astuthiodit_1, karapayneWMDE, Invadibot, 
maantietaja, ItamarWMDE, Akuckartz, Dringsim, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, KimKelting, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Wikidata-bugs, aude, zeljkofilipin, Mbch331, Jay8g, hashar, 
Isabelladantes1983, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 
786, Biggs657, Juan90264, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, 
joker88john, CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, 
Lewizho99, Maathavan, Neuronton
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2024-05-04 Thread gerritbot 
gerritbot added a comment.


  Change #769397 **abandoned** by Addshore:
  
  [mediawiki/extensions/Wikibase@master] selenium waitforTimeout to 20 seconds
  
  Reason:
  
  
  
  https://gerrit.wikimedia.org/r/769397

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, gerritbot
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, Ullasoff, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, 
karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, 
ItamarWMDE, Un1tY, Akuckartz, Dringsim, Hook696, Kent7301, joker88john, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, Lewizho99, 
Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, 
zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2022-03-21 Thread Addshore 
Addshore closed this task as "Resolved".

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, 
Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, 
Invadibot, Lalamarie69, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, 
Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, 
QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, 
Scott_WUaS, Wikidata-bugs, aude, zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2022-03-09 Thread Addshore 
Addshore added a comment.


  I centrally looked at these for quite some time and generally the only thing 
I found is that most flakiness is caused by CI being slow and us hitting 
timouts.
  If all of the timeouts were longer for CI runs (in terms of selnium tests) I 
expect that less flakiness / test failings would be seen.
  OR if CI was faster on the whole. (See T281122 
 etc)
  
  @thcipriani happy to just close this one here now, or can leave it open for 
continuing talking about the stuff you were working on!

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, 
Fernandobacasegua34, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, 
Lalamarie69, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, 
Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2022-03-09 Thread Addshore 
Addshore added a comment.


  Oh neat!
  Yeah that looks like a lovely approach.
  
  I'll run my terrible analysis one more time now and then stop, generally I 
think it's been useful.
  
addshore@contint2001:~$ find /srv/jenkins/builds/*-selenium-*/*/log 
-maxdepth 1 -mtime -7 | grep -v apache | xargs cat | grep ✖ | sort | uniq -c | 
sort -nr
  2 [Chrome 90.0.4430.212 linux #0-8]✖ "before all" hook for 
Termbox: reading
  2 [Chrome 90.0.4430.212 linux #0-7]✖ "before all" hook for 
Termbox: LicenseOverlay
  2 [Chrome 90.0.4430.212 linux #0-6]✖ "before each" hook for 
Termbox: editing
  2 [chrome 90.0.4430.212 linux #0-6]✖ "before each" hook for 
permission checks
  2 [chrome 90.0.4430.212 linux #0-5]✖ can go back from a save error
  2 [Chrome 90.0.4430.212 linux #0-5]✖ "before all" hook for 
Termbox: AnonEditWarning
  2 [chrome 90.0.4430.212 linux #0-4]✖ uses the property ID 
if no label in fallback chain found
  2 [chrome 90.0.4430.212 linux #0-4]✖ uses the label from 
the page content language
  2 [chrome 90.0.4430.212 linux #0-4]✖ uses the label from 
the English fallback language
  2 [chrome 90.0.4430.212 linux #0-4]✖ uses the label from 
a non-English fallback language
  2 [chrome 90.0.4430.212 linux #0-4]✖ shows the current 
targetValue
  2 [chrome 90.0.4430.212 linux #0-4]✖ shows the current 
targetReference
  2 [chrome 90.0.4430.212 linux #0-4]✖ opens app in OOUI dialog
  2 [chrome 90.0.4430.212 linux #0-4]✖ is shown and leads 
to login page
  2 [chrome 90.0.4430.212 linux #0-4]✖ is shown and can be 
dismissed
  2 [chrome 90.0.4430.212 linux #0-4]✖ is not shown when logged 
in
  2 [chrome 90.0.4430.212 linux #0-4]✖ indicates loading while app 
gathers its data
  2 [Chrome 90.0.4430.212 linux #0-4]✖ "before all" hook for the 
Tainted icon
  2 [chrome 90.0.4430.212 linux #0-3]✖ shows ErrorUnknown when 
launching bridge for a non-existent entity
  2 [chrome 90.0.4430.212 linux #0-3]✖ shows custom Bridge 
warning on login page
  2 [chrome 90.0.4430.212 linux #0-3]✖ can retry saving bridge from 
ErrorSaving
  2 [chrome 90.0.4430.212 linux #0-3]✖ can go back from a save 
error on desktop
  2 [Chrome 90.0.4430.212 linux #0-3]✖ can expand collapsed 
references on a protected page as unprivileged user
  2 [chrome 90.0.4430.212 linux #0-3]✖ can be relaunched from 
ErrorUnknown
  2 [chrome 90.0.4430.212 linux #0-3]✖ "before each" hook for 
when there is an edit conflict
  2 [chrome 90.0.4430.212 linux #0-3]✖ "before each" hook for 
when assertuser fails
  2 [chrome 90.0.4430.212 linux #0-2]✖ saves an updated value
  2 [chrome 90.0.4430.212 linux #0-2]✖ can save with tag
  2 [Chrome 90.0.4430.212 linux #0-1]✖ old revisions do not have an 
edit link
  2 [Chrome 90.0.4430.212 linux #0-1]✖ can add a statement using 
the keyboard
  2 [chrome 90.0.4430.212 linux #0-1]✖ "before all" hook for App
  2 [chrome 90.0.4430.212 linux #0-0]✖ unsupported datatype
  
  So some consistent ones between the last check and now would be
  
2 [Chrome 90.0.4430.212 linux #0-8]✖ "before all" hook for Termbox: 
reading
2 [Chrome 90.0.4430.212 linux #0-7]✖ "before all" hook for Termbox: 
LicenseOverlay
2 [Chrome 90.0.4430.212 linux #0-6]✖ "before each" hook for 
Termbox: editing
2 [Chrome 90.0.4430.212 linux #0-5]✖ "before all" hook for Termbox: 
AnonEditWarning
  
  Which all relate to the Wikibase termbox.
  
  
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/139725/
  
16:15:45 Error: internal_api_error_Error: [893bc2c55d0ebc26b5e731f1] 
Exception caught: Call to a member function getFullText() on null
16:15:45 at 
/workspace/src/extensions/Wikibase/node_modules/mwbot/src/index.js:262:31
  
  
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/125081/consoleFull
  
23:35:09 [Chrome 90.0.4430.212 linux #0-7] Termbox: LicenseOverlay
23:35:09 [Chrome 90.0.4430.212 linux #0-7]? is shown when clicking 
publish
23:35:09 [Chrome 90.0.4430.212 linux #0-7]✖ "before each" hook for 
Termbox: LicenseOverlay
23:35:09 [Chrome 90.0.4430.212 linux #0-7]
23:35:09 [Chrome 90.0.4430.212 linux #0-7] 1 failing (11.5s)
23:35:09 [Chrome 90.0.4430.212 linux #0-7]
23:35:09 [Chrome 90.0.4430.212 linux #0-7] 1) Termbox: LicenseOverlay 
"before each" hook for Termbox: LicenseOverlay
23:35:09 [Chrome 90.0.4430.212 linux #0-7] element 
(".wb-ui-all-entered-languages-expandable__switch") still not existing after 
1ms
23:35:09 [Chrome 90.0.4430.212 linux

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2022-03-09 Thread gerritbot 
gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, gerritbot
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, 
Fernandobacasegua34, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, 
Lalamarie69, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, 
Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2022-03-09 Thread gerritbot 
gerritbot added a comment.


  Change 769397 had a related patch set uploaded (by Addshore; author: 
Addshore):
  
  [mediawiki/extensions/Wikibase@master] selenium waitforTimeout to 20 seconds
  
  https://gerrit.wikimedia.org/r/769397

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, gerritbot
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, karapayneWMDE, 
Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2021-12-06 Thread thcipriani 
thcipriani added a comment.


  I made a script that parses all Jenkins junit files and puts them in a 
database. The database (see here ) has about 5 
million test cases  at this point.
  
  A test is flaky when it runs twice on the same code with different results, 
so I made a database query that finds any tests that both failed and passed for 
the same SHA1. Here's the jupyter notebook 
 with the 
results. Is this useful at all?

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, thcipriani
Cc: thcipriani, mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, Invadibot, 
maantietaja, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2021-11-30 Thread Addshore 
Addshore added a comment.


  This is almost not worth putting in a sheet for the last week.
  
addshore@contint2001:~$ find /srv/jenkins/builds/*-selenium-*/*/log 
-maxdepth 1 -mtime -7 | grep -v apache | xargs cat | grep ✖ | sort | uniq -c | 
sort -nr
  2 npm ERR!   ✖ Error: Command failed: /bin/sh -c ./configure 
--with-system-zlib 
--prefix="/workspace/src/extensions/VisualEditor/node_modules/optipng-bin/vendor"
 
--bindir="/workspace/src/extensions/VisualEditor/node_modules/optipng-bin/vendor"
  1 [Chrome 90.0.4430.212 linux #0-8]✖ "before each" hook for 
Termbox: reading
  1 [Chrome 90.0.4430.212 linux #0-7]✖ "before each" hook for 
Termbox: LicenseOverlay
  1 [Chrome 90.0.4430.212 linux #0-6]✖ "before each" hook for 
Termbox: editing
  1 [Chrome 90.0.4430.212 linux #0-5]✖ "before each" hook for 
Termbox: AnonEditWarning
  
  I see these failures in the last week I checked too, so will write a ticket 
to investigate them and see if we can do anything?
  
  T296710: Investigate occasionally failing before each hooks for termbox 


TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: mmodell, alaa_wmde, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, 
Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, 
_jensen, rosalieper, Scott_WUaS, thcipriani, Wikidata-bugs, aude, 
zeljkofilipin, Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T277205: Centrally look for flakey browser tests

2021-09-21 Thread Maintenance_bot 
Maintenance_bot added a project: Wikidata.

TASK DETAIL
  https://phabricator.wikimedia.org/T277205

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore, Maintenance_bot
Cc: alaa_wmde, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, Akuckartz, 
Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, 
rosalieper, Scott_WUaS, thcipriani, Wikidata-bugs, aude, zeljkofilipin, 
Mbch331, Jay8g, hashar
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org