dcausse added a comment.
It seems (still not 100% sure yet but seeing a lot of failures related to this) that the repeated failures are caused by the bad swift client we are still using for the flink ha storage, we stopped using this client for the job states (T302494 <https://phabricator.wikimedia.org/T302494>) but we haven't fully removed this client from the image (T304914 <https://phabricator.wikimedia.org/T304914>). The current immediate plan is as follow: - stop the flink jobs in codfw - stop polling codfw for wikidata max lag detection: https://gerrit.wikimedia.org/r/821753 - cleanup old checkpoints/savepoints TASK DETAIL https://phabricator.wikimedia.org/T314835 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, bking, Ottomata, dcausse, LSobanski, dr0ptp4kt, fgiunchedi, Aklapper, Raineydaz, Astuthiodit_1, AWesterinen, MatthewVernon, BTullis, karapayneWMDE, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, Muchiri124, CBogen, ItamarWMDE, Akuckartz, Di3sel1975, Chambersjay, RhinosF1, Legado_Shulgin, ReaperDawn, Nandana, Namenlos314, Conradrock, Davinaclare77, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
