[Wikidata-bugs] [Maniphest] T261937: Add CPU load and query concurrency as context to event logging from WDQS

2020-11-09 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T261937 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Zbyszko, JAllemandou, Aklapper, Gehel, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-11-09 Thread Gehel
Gehel closed subtask T256882: The streaming updater should support page undeletes as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup

[Wikidata-bugs] [Maniphest] T256882: The streaming updater should support page undeletes

2020-11-09 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T256882 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T257045: [Epic] Better understanding of WDQS users and use cases

2020-11-09 Thread Gehel
Gehel closed subtask T261937: Add CPU load and query concurrency as context to event logging from WDQS as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T257045 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: GoranSMilovanovic

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-11-09 Thread Gehel
Gehel closed subtask T265452: Add a configurable restart strategy to the streaming updater as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi

[Wikidata-bugs] [Maniphest] T263952: mwapi calls rarely return results

2020-11-09 Thread Gehel
Gehel closed this task as "Resolved". Gehel added a comment. This should be resolved, but hard to reproduce, so hard to test. Feel free to reopen if the error occurs again. TASK DETAIL https://phabricator.wikimedia.org/T263952 EMAIL PREFERENCES https://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] T265452: Add a configurable restart strategy to the streaming updater

2020-11-09 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T265452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T266318: Clarify dependencies on codehale dropwizards

2020-11-09 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T266318 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: dcausse, Aklapper, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-11-09 Thread Gehel
Gehel closed subtask T266318: Clarify dependencies on codehale dropwizards as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback

2020-11-02 Thread Gehel
Gehel added a comment. This task is about exposing wdqs1009 as a test server to the internet. This needs a new DNS entry, configuration of the caching / routing layer and communication with the traffic team. TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-11-02 Thread Gehel
Gehel removed projects: Wikibase wb_terms leftovers 2020, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). Gehel added a comment. Saneitizer is running slower than usual at the moment, this needs to be investigated first. See T266762 <https://phabricator.wikimedia.org/T266762> and r

[Wikidata-bugs] [Maniphest] T266827: New search keyword itemquality to get result from ORES

2020-11-02 Thread Gehel
Gehel added a comment. A lot more work than just creating a keyword is needed to make this work. The item quality is not indexed at the moment (but it might make sense to index it). Doing this in particular for Wikidata where data size and update rates are high needs quite a bit of thinking

[Wikidata-bugs] [Maniphest] T266828: Expose ORES item quality to WDQS

2020-11-02 Thread Gehel
Gehel added a comment. We don't want to add peripheral data into the existing WDQS storage. We might want to start another endpoint and allow federation with it. TASK DETAIL https://phabricator.wikimedia.org/T266828 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] T244341: Stop using blank nodes for encoding SomeValue and OWL constraints in WDQS

2020-10-30 Thread Gehel
Gehel added a subtask: T266470: Expose wdqs1009 to wdqs users and gather feedback. TASK DETAIL https://phabricator.wikimedia.org/T244341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: JMinor, TomT0m, Gehel, Multichill, Pfps, Mmarx

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-30 Thread Gehel
Gehel added a subtask: T266470: Expose wdqs1009 to wdqs users and gather feedback. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback

2020-10-30 Thread Gehel
Gehel added parent tasks: T244590: [Epic] Rework the WDQS updater as an event driven application, T244341: Stop using blank nodes for encoding SomeValue and OWL constraints in WDQS. TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T261119: Architecture review of Flink based WDQS Streaming Updater

2020-10-30 Thread Gehel
Gehel added a parent task: T244590: [Epic] Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T261119 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Aklapper, Zbyszko, dcausse

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-30 Thread Gehel
Gehel added a subtask: T261119: Architecture review of Flink based WDQS Streaming Updater. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-30 Thread Gehel
Gehel added a subtask: T266750: The streaming updater consumer should stop accumulating patches if it cannot handle them. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry

[Wikidata-bugs] [Maniphest] T266750: The streaming updater consumer should stop accumulating patches if it cannot handle them

2020-10-30 Thread Gehel
Gehel added a parent task: T244590: [Epic] Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T266750 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: Aklapper, dcausse, Alter

[Wikidata-bugs] [Maniphest] T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS

2020-10-30 Thread Gehel
Gehel added a subtask: T262828: Near zero downtime Data reload for WCQS. TASK DETAIL https://phabricator.wikimedia.org/T251488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: BeatEstermann, Poyekhali, Steinsplitter, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] T262828: Near zero downtime Data reload for WCQS

2020-10-30 Thread Gehel
Gehel added a parent task: T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS. TASK DETAIL https://phabricator.wikimedia.org/T262828 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Gehel, Bugreporter, Aklapper

[Wikidata-bugs] [Maniphest] T266702: Move WDQS UI to microsites

2020-10-29 Thread Gehel
Gehel added a comment. I don't like having services proxying each others internally. Both `edge->wdqs->static` and `edge-static-wdqs` are adding accidental complexity to the architecture. This should really be a traffic routing question, wdqs and wdqs-ui are to separate se

[Wikidata-bugs] [Maniphest] T259588: Reload categories once 1.36.0-wmf.3 is running on all groups

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T259588 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: dcausse

[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T264006 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Mstyles, Gehel

[Wikidata-bugs] [Maniphest] T258269: Add query result to the current WDQS event logging

2020-10-28 Thread Gehel
Gehel lowered the priority of this task from "High" to "Medium". TASK DETAIL https://phabricator.wikimedia.org/T258269 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] T264447: Examine cases where Blazegraph generates results that timeout and don’t make it back to the user

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T264447 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: CBogen, Lydia

[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T266495 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: elukey, EB

[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T265526 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, Gehe

[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T265512 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, Gehe

[Wikidata-bugs] [Maniphest] T258269: Add query result to the current WDQS event logging

2020-10-28 Thread Gehel
Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T258269 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: J

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-26 Thread Gehel
Gehel closed subtask T256875: The streaming updater should support page deletions as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup

[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions

2020-10-26 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles, Gehel Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-26 Thread Gehel
Gehel closed subtask T256949: The streaming updater should support suppressed deletes as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup

[Wikidata-bugs] [Maniphest] T256949: The streaming updater should support suppressed deletes

2020-10-26 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles, Gehel Cc: Mstyles, Bugreporter, dcausse, Aklapper, Alter-paule, Beast1978, CBogen, Un1tY,

[Wikidata-bugs] [Maniphest] T262009: Add an entry in the WDQS Runbook on killer queries

2020-10-26 Thread Gehel
Gehel closed this task as "Resolved". Gehel claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T262009 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, RKemper, Gehel, CBogen, Akuckartz, Nandana, Namenlos3

[Wikidata-bugs] [Maniphest] T265868: wdqs2002 out of disk space

2020-10-26 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T265868 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: RKemper, dcausse, Gehel, Aklapper, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T265868: wdqs2002 out of disk space

2020-10-19 Thread Gehel
Gehel created this task. Gehel added projects: Wikidata-Query-Service, Discovery-Search (Current work). Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION The journal has grown to 1.3TB: gehel@wdqs2002:~$ ls -lh /srv

[Wikidata-bugs] [Maniphest] T265868: wdqs2002 out of disk space

2020-10-19 Thread Gehel
Gehel moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. Gehel claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T265868 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T265290: Review the federation input process for WDQS

2020-10-12 Thread Gehel
Gehel moved this task from All WDQS-related tasks to Current work on the Wikidata-Query-Service board. Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T265290 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL

[Wikidata-bugs] [Maniphest] T264642: Log Formatting for WDQS

2020-10-12 Thread Gehel
Gehel added a comment. It should be relatively easy to edit the logback.xml configuration file. I'm not entirely sure how this would be done in the context of the docker images, maybe @Addshore has a good idea. See documentation: http://logback.qos.ch/manual/layouts.html

[Wikidata-bugs] [Maniphest] T263427: Unable to process a particular wikibase dump using munge.sh (localised namespace name)

2020-10-12 Thread Gehel
Gehel triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T263427 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: dcausse

[Wikidata-bugs] [Maniphest] T264659: Update BAG & BRT SPARQL endpoint in the whitelist

2020-10-12 Thread Gehel
Gehel removed a project: Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T264659 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Denengelse, RhinosF1, Aklapper, Multichill, CBogen, Akuckartz, CptViraj, darthmon_wmde

[Wikidata-bugs] [Maniphest] T264659: Update BAG & BRT SPARQL endpoint in the whitelist

2020-10-12 Thread Gehel
Gehel moved this task from needs triage to Current work on the Discovery-Search board. Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T264659 WORKBOARD https://phabricator.wikimedia.org/project/board/1849/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T261337: Evaluate using the query manipulator in a production environment

2020-10-12 Thread Gehel
Gehel added a comment. We'll discuss this during the weekly Search Chat on Wednesday Oct 15th. TASK DETAIL https://phabricator.wikimedia.org/T261337 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] T265290: Review the federation input process for WDQS

2020-10-12 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a user of WDQS, I want to have an easy way to submit additional federation endpoints

[Wikidata-bugs] [Maniphest] T257045: [Epic] Better understanding of WDQS users and use cases

2020-10-12 Thread Gehel
Gehel closed subtask T261841: Tag WDQS query log with the source of the query (UI vs direct access) as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T257045 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: GoranSMilovanovic

[Wikidata-bugs] [Maniphest] T261841: Tag WDQS query log with the source of the query (UI vs direct access)

2020-10-12 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T261841 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: CBogen, JAllemandou, Aklapper, Gehel, Alter-paule, Beast1978, Un1tY, Akuckart

[Wikidata-bugs] [Maniphest] T255399: Prepare wdqs1009 to run the streaming updater

2020-10-08 Thread Gehel
Gehel added a comment. Now that SSD has been replaced in wdqs1009 (T263125 <https://phabricator.wikimedia.org/T263125>), we can restart this data import. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS

2020-10-06 Thread Gehel
Gehel closed this task as "Resolved". Gehel claimed this task. Gehel added a comment. WCQS has been available on WMCS for some time with no major issues. Let's close this! TASK DETAIL https://phabricator.wikimedia.org/T251488 EMAIL PREFERENCES https://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] T141602: [Objective Fiscal 19-20/Q4] (9) Provide a Proof of Concept SPARQL endpoint in support of SDoC project

2020-10-06 Thread Gehel
Gehel closed subtask T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T141602 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: danshick-wmde, Gamaliel

[Wikidata-bugs] [Maniphest] T257058: [Epic] Review / Improve Search Platform team documentation

2020-10-06 Thread Gehel
Gehel renamed this task from "Review / Improve Search Platform team documentation" to "[Epic] Review / Improve Search Platform team documentation". TASK DETAIL https://phabricator.wikimedia.org/T257058 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/pa

[Wikidata-bugs] [Maniphest] T244341: Stop using blank nodes for encoding SomeValue and OWL constraints in WDQS

2020-10-06 Thread Gehel
Gehel removed a project: Epic. TASK DETAIL https://phabricator.wikimedia.org/T244341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: JMinor, TomT0m, Gehel, Multichill, Pfps, Mmarx, Dipsacus_fullonum, Luitzen, VladimirAlexiev

[Wikidata-bugs] [Maniphest] T244341: Stop using blank nodes for encoding SomeValue and OWL constraints in WDQS

2020-10-06 Thread Gehel
Gehel removed a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T244341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: JMinor, TomT0m, Gehel, Multichill, Pfps, Mmarx, Dipsacus_fullonum, Luitzen

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2020-10-06 Thread Gehel
Gehel renamed this task from "EPIC: Rework the WDQS updater as an event driven application" to "[Epic] Rework the WDQS updater as an event driven application". TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedi

[Wikidata-bugs] [Maniphest] T263491: [EPIC] WDQS behind API Gateway

2020-10-06 Thread Gehel
Gehel triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T263491 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, Ge

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-05 Thread Gehel
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: tfmorris, Pintoch, CBogen, EBernhardson

[Wikidata-bugs] [Maniphest] T264404: Determine a way of separating truthy queries

2020-10-05 Thread Gehel
Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T264404 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] T264447: Examine cases where Blazegraph generates results that timeout and don’t make it back to the user

2020-10-05 Thread Gehel
Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T264447 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] T263427: Unable to process a particular wikibase dump using munge.sh (localised namespace name)

2020-10-05 Thread Gehel
Gehel added a project: Wikidata-Query-Service. TASK DETAIL https://phabricator.wikimedia.org/T263427 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: dcausse, Aklapper, Nikerabbit, CBogen, Samantha_Alipio_WMDE, Akuckartz, darthmon_wmde

[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)

2020-10-05 Thread Gehel
Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] T262896: triples should be marked as "linked/unlinked shared statements"

2020-10-05 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T262896 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-10-05 Thread Gehel
Gehel closed subtask T262896: http://wikiba.se/ontology#wikiGroup; triples should be marked as linked/unlinked shared statements as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] T259115: Import wikidata ttl dumps in a hive table

2020-10-05 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T259115 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T264164: Cleanup broken dumps in /wikidatawiki/entities/20200921/

2020-10-05 Thread Gehel
Gehel closed this task as "Resolved". Gehel claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T264164 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: ArielGlenn, Gehel, dcausse, jannee_e, CBogen, Akuckartz, dar

[Wikidata-bugs] [Maniphest] T258754: Define SLOs and error budget for WDQS

2020-10-05 Thread Gehel
Gehel added a comment. Note that a similar discussion was already started on T199228 <https://phabricator.wikimedia.org/T199228>. It was closed, waiting for architectural changes to be implemented first. TASK DETAIL https://phabricator.wikimedia.org/T258754 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T264447: Examine cases where Blazegraph generates results that timeout and don’t make it back to the user

2020-10-02 Thread Gehel
Gehel created this task. Gehel added projects: Wikidata, Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As someone working on WDQS, I want to better understand the limits of the service, the use cases that we don't support but that are consuming

[Wikidata-bugs] [Maniphest] T261337: Evaluate using the query manipulator in a production environment

2020-10-02 Thread Gehel
Gehel added a subtask: T264406: Run an experiment with Queripulator with either production data flows, or with query data extracted from production logs. TASK DETAIL https://phabricator.wikimedia.org/T261337 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] T264406: Run an experiment with Queripulator with either production data flows, or with query data extracted from production logs

2020-10-02 Thread Gehel
Gehel added a parent task: T261337: Evaluate using the query manipulator in a production environment. TASK DETAIL https://phabricator.wikimedia.org/T264406 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T264406: Run an experiment with Queripulator with either production data flows, or with query data extracted from production logs

2020-10-02 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a developer of WDQS, I want to understand the impact and feasibility of deploying Queripulator

[Wikidata-bugs] [Maniphest] T261337: Evaluate using the query manipulator in a production environment

2020-10-02 Thread Gehel
Gehel added a subtask: T264405: Determine how to provide a way of separating linked data fragments from WDQS. TASK DETAIL https://phabricator.wikimedia.org/T261337 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] T264405: Determine how to provide a way of separating linked data fragments from WDQS

2020-10-02 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a user of WDQS, I want my simpler LDF based queries to be executed efficiently and not slowed down

[Wikidata-bugs] [Maniphest] T264405: Determine how to provide a way of separating linked data fragments from WDQS

2020-10-02 Thread Gehel
Gehel added a parent task: T261337: Evaluate using the query manipulator in a production environment. TASK DETAIL https://phabricator.wikimedia.org/T264405 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T264404: Determine a way of separating truthy queries

2020-10-02 Thread Gehel
Gehel added a parent task: T261337: Evaluate using the query manipulator in a production environment. TASK DETAIL https://phabricator.wikimedia.org/T264404 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T261337: Evaluate using the query manipulator in a production environment

2020-10-02 Thread Gehel
Gehel added a subtask: T264404: Determine a way of separating truthy queries. TASK DETAIL https://phabricator.wikimedia.org/T261337 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: EBernhardson, LucasWerkmeister, Jakob_WMDE, Addshore

[Wikidata-bugs] [Maniphest] T264404: Determine a way of separating truthy queries

2020-10-02 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a user of WDQS, I want my simpler truthy queries to not be impacted by the load of more complex

[Wikidata-bugs] [Maniphest] T264402: Put dummy data in front of Queripulator to analyze how queries are routed

2020-10-02 Thread Gehel
Gehel added a parent task: T261337: Evaluate using the query manipulator in a production environment. TASK DETAIL https://phabricator.wikimedia.org/T264402 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Aklapper, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T264402: Put dummy data in front of Queripulator to analyze how queries are routed

2020-10-02 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a developer of WDQS, I want to have data on the potential use case of Queriputalor to inform future

[Wikidata-bugs] [Maniphest] T261337: Evaluate using the query manipulator in a production environment

2020-10-02 Thread Gehel
Gehel added a subtask: T264402: Put dummy data in front of Queripulator to analyze how queries are routed. TASK DETAIL https://phabricator.wikimedia.org/T261337 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: EBernhardson, LucasWerkmeister

[Wikidata-bugs] [Maniphest] T264164: Cleanup broken dumps in /wikidatawiki/entities/20200921/

2020-10-01 Thread Gehel
Gehel set the point value for this task to "1". Gehel set Final Story Points to 1. TASK DETAIL https://phabricator.wikimedia.org/T264164 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: ArielGlenn, Gehel, dcausse, jannee_e, CBogen,

[Wikidata-bugs] [Maniphest] T264164: Cleanup broken dumps in /wikidatawiki/entities/20200921/

2020-10-01 Thread Gehel
Gehel added a comment. The dump was already removed from dumpsdata1002 (or I can't find it). Problematic dumps have been removed from labstore100[67]. TASK DETAIL https://phabricator.wikimedia.org/T264164 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] T264164: Cleanup broken dumps in /wikidatawiki/entities/20200921/

2020-10-01 Thread Gehel
Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T264164 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: ArielGlenn, Gehel, dcausse, jannee_e, CBogen, Akuckartz, darthmon_wmde, Nandana

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-30 Thread Gehel
Gehel added a comment. Note that it is trivial to reimage this server, so feel free to nuke it instead of rebuilding the raid if it's easier (or use this as a learning opportunity to automate it next time). TASK DETAIL https://phabricator.wikimedia.org/T263125 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2020-09-30 Thread Gehel
Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Addshore, dcausse, Aklapper, NavinRizwi, CBogen, Akuckartz, darthmon_wmde, DannyS712, Nandana

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2020-09-30 Thread Gehel
Gehel added a comment. With our new streaming updater, the constraint are going to change. Let's revisit once the streaming updater is ready. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2020-09-30 Thread Gehel
Gehel lowered the priority of this task from "High" to "Medium". TASK DETAIL https://phabricator.wikimedia.org/T241128 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] T206560: [Epic] Evaluate alternatives to Blazegraph

2020-09-30 Thread Gehel
Gehel raised the priority of this task from "Medium" to "High". TASK DETAIL https://phabricator.wikimedia.org/T206560 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] T187424: Support queued SPARQL request

2020-09-30 Thread Gehel
Gehel raised the priority of this task from "Low" to "Medium". TASK DETAIL https://phabricator.wikimedia.org/T187424 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-30 Thread Gehel
Gehel added projects: DC-Ops, ops-eqiad. Gehel reassigned this task from Gehel to wiki_willy. Gehel added a subscriber: wiki_willy. Gehel added a comment. Restricted Application added a project: Operations. `smart-data-dump` shows a few errors for sdb: # HELP

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-30 Thread Gehel
Gehel added a comment. One error found by badblocks at block 65: From block 0 to 1464897535 Testing with pattern 0xaa: done Reading and comparing: done Testing with pattern

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-29 Thread Gehel
Gehel added a comment. Running check via `sudo badblocks -v -o badblocks -w -f /dev/mapper/wdqs1009--vg- data &> badblocks.log`. This is still going through the raid layer, but should exercise the disks enough to get some confidence that it works. TASK DETAIL

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-29 Thread Gehel
Gehel claimed this task. Gehel moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T263125 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T263125: Check for errors on wdqs1009 disks

2020-09-28 Thread Gehel
Gehel added a comment. In T263125#6499600 <https://phabricator.wikimedia.org/T263125#6499600>, @EBernhardson wrote: > A historical unix program, shipped with many distros, for writing patterns to a disk and reading them back to verify correctness: > > https://wik

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-09-28 Thread Gehel
Gehel closed subtask T258240: Refactor Options handling in Streaming Updater as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Thadguidry, tfmorris, revi, Ladsgroup

[Wikidata-bugs] [Maniphest] T261119: Architecture review of Flink based WDQS Streaming Updater

2020-09-28 Thread Gehel
Gehel closed subtask T258240: Refactor Options handling in Streaming Updater as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T261119 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Aklapper, Zbyszko, dcausse, Gehel

[Wikidata-bugs] [Maniphest] T258240: Refactor Options handling in Streaming Updater

2020-09-28 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T258240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: dcausse, Aklapper, Zbyszko, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlo

[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)

2020-09-28 Thread Gehel
Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As on operator of WDQS / WCQS, I want to deploy Flink in a way that is robust so that we can deploy

[Wikidata-bugs] [Maniphest] T261125: Allow domain wikibooks.org from wdqs mwapi service

2020-09-28 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T261125 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T259637: Gather information about the volume of queries on WCQS

2020-09-28 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T259637 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Zbyszko, EBernhardson, Aklapper, dcausse, CBogen, Alter-paule, Beast1978, Un1tY,

[Wikidata-bugs] [Maniphest] T261097: WDQS Categories reload is failing on thankyouwiki

2020-09-28 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T261097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: dcausse, RKemper, Gehel, Aklapper, lmata, CBogen, Akuckartz, darthmon_wmde, Lega

<    9   10   11   12   13   14   15   16   17   18   >