[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: Mstyles, dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Suran38, Biggs657, Invadibo

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

gerritbot added a comment. Change 671204 **merged** by jenkins-bot: [operations/deployment-charts@master] rdf-streaming-updater: switch to H/A session-cluster https://gerrit.wikimedia.org/r/671204 TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://p

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Gehel reassigned this task from Mstyles to dcausse. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: Mstyles, dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Invadibot, La

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

gerritbot added a comment. Change 679519 **abandoned** by DCausse: [operations/deployment-charts@master] rdf-streaming-updater: enable HA capability Reason: squashed into parent https://gerrit.wikimedia.org/r/679519 TASK DETAIL https://phabricator.wikimedia.org/T273098

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

gerritbot added a comment. Change 671204 had a related patch set uploaded (by DCausse; author: Mstyles): [operations/deployment-charts@master] rdf-streaming-updater: switch to H/A session-cluster https://gerrit.wikimedia.org/r/671204 TASK DETAIL https://phabricator.wikimedia.org/T

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Mstyles added a comment. So one problem with implementing HA without using the session cluster (T280166 ) is that Flink can't properly start the Streaming Updater job locally, since the jar needs to be stored in the storage directory. I'm not sure o

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles, gerritbot Cc: Mstyles, dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Invadibot, MPham

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

gerritbot added a comment. Change 679519 had a related patch set uploaded (by Mstyles; author: Mstyles): [operations/deployment-charts@master] rdf-streaming-updater: enable HA capability https://gerrit.wikimedia.org/r/679519 TASK DETAIL https://phabricator.wikimedia.org/T273098 E

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Mstyles added a comment. 3 config maps are created by the pods, `dispatcher-leader`, `resourcemanager-leader`, and `restserver-leader`. There aren't any references from helm to these configmaps, so when helm is updated, it shouldn't affect the configmaps. TASK DETAIL https://phabricator

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

dcausse added a comment. In T273098#6997661 , @JMeybohm wrote: > I do see that using the configmap election method is appealing as it is build in and does not require additional software to function. Unfortunately I was not able to unde

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

JMeybohm added a comment. I do see that using the configmap election method is appealing as it is build in and does not require additional software to function. Unfortunately I was not able to understand (by briefly reading the docs) if this uses a separate configmap or the one that is actua

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Mstyles edited subscribers, added: jijiki, JMeybohm; removed: Mstyles. Mstyles added a comment. The changes for HA are mostly changing a few lines of the config file. The part that is unclear is whether we can get a service account for pods that allow them to do CRUD operations for configmaps

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq

[Wikidata-bugs] [Maniphest] T273098: High Availability Flink

Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION With the latest updated of Flink (1.12), High Availability is now supported for F