dcausse created this task. dcausse added projects: Wikidata, Wikidata-Query-Service, serviceops, Data-Platform-SRE. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: wmde-wikidata-tech.
TASK DESCRIPTION As part of the work to split the WDQS graph we will need to populate 4 new topics: - eqiad.rdf-streaming-updater.mutation-main - codfw.rdf-streaming-updater.mutation-main - eqiad.rdf-streaming-updater.mutation-scholarly - codfw.rdf-streaming-updater.mutation-scholarly The expected size of both added should not exceed the size of `eqiad.rdf-streaming-updater.mutation` which is around 17Gb (51Gb <https://grafana-rw.wikimedia.org/d/000000234/kafka-by-topic?orgId=1&refresh=5m&var-datasource=eqiad%20prometheus%2Fops&var-kafka_cluster=main-eqiad&var-kafka_broker=All&var-topic=eqiad.rdf-streaming-updater.mutation&var-topic=codfw.rdf-streaming-updater.mutation&from=now-7d&to=now> including replication). Similarly for the rate of messages. Because topic mirroring this means that an additional 100Gb per cluster is required (+100Gb on main-eqiad and +100Gb on main-codfw). These topics must have the following characteristics: - a single partition - retention of 4 weeks AC: [ ] get sign off from #serviceops <https://phabricator.wikimedia.org/tag/serviceops/> [ ] topics are created in both clusters with proper settings TASK DETAIL https://phabricator.wikimedia.org/T367510 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Danny_Benjafield_WMDE, Kappakayala, S8321414, Clement_Goubert, Astuthiodit_1, AWesterinen, Arnoldokoth, BTullis, karapayneWMDE, Invadibot, maantietaja, wkandek, JMeybohm, ItamarWMDE, Akuckartz, Dringsim, Nandana, Namenlos314, jijiki, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, KimKelting, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
