[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: EBernhardson, RShigapov, dcausse, Aklapper, Fernandobacasegua34, Astuthiodit_1, 786, Suran3

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 772906 **merged** by jenkins-bot: [wikimedia/discovery/analytics@master] rdf_streaming_updater_reconcile: set proper start dates https://gerrit.wikimedia.org/r/772906 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 772906 had a related patch set uploaded (by DCausse; author: DCausse): [wikimedia/discovery/analytics@master] rdf_streaming_updater_reconcile: set proper start dates https://gerrit.wikimedia.org/r/772906 TASK DETAIL https://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Gehel added a parent task: T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: EBernhardson, RShigapov, dcau

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse added a comment. Deployment of this feature has been stopped due to T302340 . TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: EBe

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 763514 **merged** by jenkins-bot: [wikimedia/discovery/analytics@master] Update rdf-spark-tools artifact and set proper start dates... https://gerrit.wikimedia.org/r/763514 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 763514 had a related patch set uploaded (by DCausse; author: DCausse): [wikimedia/discovery/analytics@master] Update rdf-spark-tools artifact and set proper start dates... https://gerrit.wikimedia.org/r/763514 TASK DETAIL https://phabricator.wikim

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 761647 **merged** by DCausse: [wikidata/query/deploy@master] Add configuration for the reconciliation topic https://gerrit.wikimedia.org/r/761647 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimed

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 761647 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/deploy@master] Add configuration for the reconciliation topic https://gerrit.wikimedia.org/r/761647 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

EBernhardson added a comment. Airflow DAG has been deployed. I have left it turned off for now, when ready someone will need to enable it (and potentially update the start_date). TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/set

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 753791 **merged** by jenkins-bot: [wikimedia/discovery/analytics@master] Schedule rdf-streaming-updater reconciliation job https://gerrit.wikimedia.org/r/753791 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://pha

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2022-02-02T01:03:13Z] Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:753788|rdf-streaming-updater: add the reconciliation stream (T279541 )]] (duration: 00m 49s)

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 753788 **merged** by jenkins-bot: [operations/mediawiki-config@master] rdf-streaming-updater: add the reconciliation stream https://gerrit.wikimedia.org/r/753788 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://ph

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 758926 **merged** by jenkins-bot: [operations/deployment-charts@master] eventgate-main: update image to 2022-02-01-141357-production https://gerrit.wikimedia.org/r/758926 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES h

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 758926 had a related patch set uploaded (by DCausse; author: DCausse): [operations/deployment-charts@master] eventgate-main: update image to 2022-02-01-141357-production https://gerrit.wikimedia.org/r/758926 TASK DETAIL https://phabricator.wikimed

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 758827 **merged** by Ottomata: [eventgate-wikimedia@master] Bump secondary schema repo to 52e2206 https://gerrit.wikimedia.org/r/758827 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/set

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 758827 had a related patch set uploaded (by DCausse; author: DCausse): [eventgate-wikimedia@master] Bump secondary schema repo to 52e2206 https://gerrit.wikimedia.org/r/758827 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 752184 **merged** by jenkins-bot: [wikidata/query/rdf@master] Add a spark job to emit reconciliation events https://gerrit.wikimedia.org/r/752184 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimed

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739916 **merged** by jenkins-bot: [wikidata/query/rdf@master] producer: add support for reconciliation https://gerrit.wikimedia.org/r/739916 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 757667 **merged** by Ottomata: [operations/deployment-charts@master] eventgate-main: update image to 2022-01-27-143826-production https://gerrit.wikimedia.org/r/757667 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES http

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 757667 had a related patch set uploaded (by DCausse; author: DCausse): [operations/deployment-charts@master] eventgate-main: update image to 2022-01-27-143826-production https://gerrit.wikimedia.org/r/757667 TASK DETAIL https://phabricator.wikimed

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 757665 **merged** by Ottomata: [eventgate-wikimedia@master] Bump secondary schema repo to 117c3fa https://gerrit.wikimedia.org/r/757665 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/set

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 757665 had a related patch set uploaded (by DCausse; author: DCausse): [eventgate-wikimedia@master] Bump secondary schema repo to 117c3fa https://gerrit.wikimedia.org/r/757665 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 756536 **merged** by jenkins-bot: [schemas/event/secondary@master] rdf_streaming_updater: add a reconcile event schema https://gerrit.wikimedia.org/r/756536 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabric

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 737429 **merged** by jenkins-bot: [schemas/event/secondary@master] rdf-streaming-updater: add a \"reconcile\" operation https://gerrit.wikimedia.org/r/737429 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabri

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 740109 **abandoned** by DCausse: [schemas/event/primary@master] rdf_streaming_updater: add a reconcile event schema Reason: will continue using the secondary repo https://gerrit.wikimedia.org/r/740109 TASK DETAIL https://phabricator.wiki

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 756536 had a related patch set uploaded (by DCausse; author: DCausse): [schemas/event/secondary@master] rdf_streaming_updater: add a reconcile event schema https://gerrit.wikimedia.org/r/756536 TASK DETAIL https://phabricator.wikimedia.org/T279541

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 753791 had a related patch set uploaded (by DCausse; author: DCausse): [wikimedia/discovery/analytics@master] [WIP] Schedule rdf-streaming-updater reconciliation job https://gerrit.wikimedia.org/r/753791 TASK DETAIL https://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 753788 had a related patch set uploaded (by DCausse; author: DCausse): [operations/mediawiki-config@master] rdf-streaming-updater: add the reconciliation stream https://gerrit.wikimedia.org/r/753788 TASK DETAIL https://phabricator.wikimedia.org/T2

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 752184 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/rdf@master] Add a spark job to emit reconciliation events https://gerrit.wikimedia.org/r/752184 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREF

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Gehel closed subtask T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739915 **merged** by jenkins-bot: [wikidata/query/rdf@master] Add a reconcile event platform event https://gerrit.wikimedia.org/r/739915 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/se

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739105 **merged** by jenkins-bot: [wikidata/query/rdf@master] consumer: add support for reconciliation https://gerrit.wikimedia.org/r/739105 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 740109 had a related patch set uploaded (by DCausse; author: DCausse): [schemas/event/primary@master] rdf_streaming_updater: add a reconcile event schema https://gerrit.wikimedia.org/r/740109 TASK DETAIL https://phabricator.wikimedia.org/T279541

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739916 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/rdf@master] [WIP] producer: add support for reconciliation https://gerrit.wikimedia.org/r/739916 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PRE

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739915 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/rdf@master] Add a reconcile event platform event https://gerrit.wikimedia.org/r/739915 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 739105 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/rdf@master] consumer: add support for reconciliation https://gerrit.wikimedia.org/r/739105 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENC

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 737436 **merged** by jenkins-bot: [wikidata/query/rdf@master] model: add a \"reconcile\" operation to MutationEventData https://gerrit.wikimedia.org/r/737436 TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabri

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 737436 had a related patch set uploaded (by DCausse; author: DCausse): [wikidata/query/rdf@master] model: add a \"reconcile\" operation to MutationEventData https://gerrit.wikimedia.org/r/737436 TASK DETAIL https://phabricator.wikimedia.org/T27954

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, gerritbot Cc: RShigapov, dcausse, Aklapper, 786, Suran38, Biggs657, Invadibot, Lalamarie69, MP

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

gerritbot added a comment. Change 737429 had a related patch set uploaded (by DCausse; author: DCausse): [schemas/event/secondary@master] [WIP] rdf-streaming-updater: add a \"reconcile\" operation https://gerrit.wikimedia.org/r/737429 TASK DETAIL https://phabricator.wikimedia.org/

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse claimed this task. dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T279541 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimed

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse added a subtask: T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/pa

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

MPhamWMF set the point value for this task to "13". TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Na

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

MPhamWMF moved this task from Scaling to Current work on the Wikidata-Query-Service board. MPhamWMF added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T279541 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES http

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Gehel removed a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

Gehel moved this task from Scaling to Current work on the Wikidata-Query-Service board. Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T279541 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://ph

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

MPhamWMF updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, La

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

MPhamWMF triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlo

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse added a parent task: T244590: [Epic] Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPham

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a maintainer of WDQS I want the streaming updater to be able to reconcile a wikibase item so tha