[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117

2022-02-01 Thread dcausse
dcausse added a comment. @ArielGlenn no problem! :) When these dumps fail we're informed couple days after and it might be interesting for us to be pro-active about that but not sure we have enough knowledge of the dump process/infra to be super useful in case of failures, if you feel

[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117

2022-02-01 Thread dcausse
dcausse closed this task as "Invalid". dcausse added a comment. Thanks for checking! TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: ArielGlenn, Aklapper, JAllemandou, A

[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117

2022-01-28 Thread dcausse
dcausse renamed this task from "TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00" to "Missing Wikidata RDF (ttl and nt) dumps for 20220117". dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL

[Wikidata-bugs] [Maniphest] T300240: TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00

2022-01-28 Thread dcausse
dcausse added a comment. RDF dumps seem absent from https://dumps.wikimedia.org/wikidatawiki/entities/20220117/ but seems to be there again on 20220124. Everything is OK now so I suspect a transient failure. Pinging #dumps-generation <https://phabricator.wikimedia.org/tag/dumps-generat

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-27 Thread dcausse
dcausse closed this task as "Resolved". dcausse added a comment. Thanks for the report, blazegraph died I restarted it, should be available again now. TASK DETAIL https://phabricator.wikimedia.org/T297454 EMAIL PREFERENCES https://phabricator.wikimedia.org/sett

[Wikidata-bugs] [Maniphest] T300240: TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00

2022-01-27 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As reported by airflow, a sensor is timing out on: [2022-01-27 04:41:24,843] {hdfs_cli.py:71} INFO - Checking marker at hdfs://analytics

[Wikidata-bugs] [Maniphest] T299460: Evaluate Apache Jena

2022-01-24 Thread dcausse
dcausse added a comment. Sorry for the confusion that the rename I did of this task caused. Just to bring clarity on my reasoning as a maintainer of the wikidata query service stack as to why being specific on TDB2 might be helpful: - Some components of Jena are already being used (i.e

[Wikidata-bugs] [Maniphest] T299460: Evaluate Apache Jena TDB2

2022-01-21 Thread dcausse
dcausse renamed this task from "Evaluate Apache Jena" to "Evaluate Apache Jena TDB2". dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T299460 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: d

[Wikidata-bugs] [Maniphest] T299290: Unexpected behavior in federated queries in WDQS

2022-01-17 Thread dcausse
dcausse added a comment. WDQS receives `Status Code=502, Status Line=Bad Gateway, Response=` from lingualibre servers. I'm not totally sure to understand why it's failing esp. why Shopox is generating a query that is accepted there and why it may sometimes succeed from wdqs when varying

[Wikidata-bugs] [Maniphest] T281468: Automatic SI unit conversion not working on Commons SPARQL engine

2022-01-11 Thread dcausse
dcausse assigned this task to Ladsgroup. dcausse moved this task from Ready for Development to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. I think it's working properly now TASK DETAIL https://phabricator.wikimedia.org/T281468 WORKBOARD https

[Wikidata-bugs] [Maniphest] T281468: Automatic SI unit conversion not working on Commons SPARQL engine

2022-01-11 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T281468 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Ladsgroup, WMDE-leszek, Addshore, Aklapper, CBogen, Lucas_Werkmeister_WMDE, Multichill

[Wikidata-bugs] [Maniphest] T262265: Provide real-time updates for WCQS

2022-01-05 Thread dcausse
dcausse added a subtask: T298622: Adapt EntityRevisionMapGenerator for wcqs. TASK DETAIL https://phabricator.wikimedia.org/T262265 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EBernhardson, dcausse Cc: Back_ache, So9q, Salgo60, Gehel, Aklapper

[Wikidata-bugs] [Maniphest] T298622: Adapt EntityRevisionMapGenerator for wcqs

2022-01-05 Thread dcausse
dcausse added a parent task: T262265: Provide real-time updates for WCQS. TASK DETAIL https://phabricator.wikimedia.org/T298622 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86

[Wikidata-bugs] [Maniphest] T298622: Adapt EntityRevisionMapGenerator for wcqs

2022-01-05 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION This spark job is required for building the initial state of the streaming updater, it must be adapted for commons. AC: - add new

[Wikidata-bugs] [Maniphest] T240334: Evaluate adding all/some textual properties to the text field

2022-01-03 Thread dcausse
dcausse added a comment. Forwarding a suggestion made on https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_Search: > It would be interesting to be able to search for street address (P6375)-values, e.g. Special:Search/Getreidegasse Salzburg should find Q37970

[Wikidata-bugs] [Maniphest] T297870: WDQS Streaming Updater fails with Timeout expired after 60000milliseconds while awaiting InitProducerId

2021-12-16 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION This error causes the pipeline to restart and might trigger the latency alert //WdqsStreamingUpdaterFlinkProcessingLatencyIsHigh//. It was seen

[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata

2021-11-29 Thread dcausse
dcausse merged a task: T295941: WDQS Data drift. dcausse added subscribers: William_Avery, dcausse. TASK DETAIL https://phabricator.wikimedia.org/T294076 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, William_Avery, RShigapov

[Wikidata-bugs] [Maniphest] T295941: WDQS Data drift

2021-11-29 Thread dcausse
dcausse closed this task as a duplicate of T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T295941 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse

[Wikidata-bugs] [Maniphest] T295941: WDQS Data drift

2021-11-29 Thread dcausse
dcausse added a comment. @William_Avery thanks for the report this is very helpful. I found almost all listed items in the "fetch failures" data set and these will be corrected once we have T279541 <https://phabricator.wikimedia.org/T279541> in place.

[Wikidata-bugs] [Maniphest] T280485: Additional capacity on the k8s Flink cluster for WCQS updater

2021-11-17 Thread dcausse
dcausse added a comment. small precision: If we reuse the same cluster (same k8s namescape): - it's 3 more pods at 2.1G ram, cpu: 1000m each If we reuse a separate cluster (new k8s namescape): - add a pod at 1.6G, cpu: 500m to the 3 pods mentioned above TASK DETAIL https

[Wikidata-bugs] [Maniphest] T293063: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes

2021-11-09 Thread dcausse
dcausse added a comment. In T293063#7491903 <https://phabricator.wikimedia.org/T293063#7491903>, @JMeybohm wrote: > @dcausse IIRC we said that "something in the areas of hours" would be considered a "short maintenance" and thus would not need any addition

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

2021-11-08 Thread dcausse
dcausse claimed this task. dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T279541 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-27 Thread dcausse
dcausse added a comment. In T293195#7459268 <https://phabricator.wikimedia.org/T293195#7459268>, @Ottomata wrote: > I was about to merge that today but then thought that your suggestion to ensure that properties validate with the additionalProperties stuff would be good to

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-26 Thread dcausse
dcausse added a comment. This is blocked on https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/629406 which is required to support the new pattern (additionalProperties + properties). @Ottomata is there anything we could do help unblock the work on your refinery patch? TASK

[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata

2021-10-26 Thread dcausse
dcausse moved this task from In Progress to Waiting on the Discovery-Search (Current work) board. dcausse added a comment. Thanks for the report this is very helpful. In the two updates you mention here were missed by the new updater but both of these were properly identified

[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater

2021-10-26 Thread dcausse
dcausse added a subtask: T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure

2021-10-26 Thread dcausse
dcausse added a parent task: T279541: Add a reconciliation strategy to the wdqs streaming updater. TASK DETAIL https://phabricator.wikimedia.org/T294361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, EChetty, MPhamWMF

[Wikidata-bugs] [Maniphest] T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure

2021-10-26 Thread dcausse
dcausse created this task. dcausse added projects: Analytics, Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION While investigating missed updates on the WDQS streaming updater I looked at our flink side-outputs that record all the failures happening

[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata

2021-10-26 Thread dcausse
dcausse claimed this task. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T294076 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams

2021-10-22 Thread dcausse
dcausse added a subscriber: Ottomata. dcausse added a project: EventStreams. dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Ottomata, Aklapper

[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams

2021-10-22 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst

[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams

2021-10-22 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a consumer of the wikidata content I want to be able to have access the same RDF data the WMF WDQS servers use to perform their live updates so

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2021-10-22 Thread dcausse
dcausse closed subtask T266321: Determine flink metrics configuration and backend when running from k8s as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc

[Wikidata-bugs] [Maniphest] T266321: Determine flink metrics configuration and backend when running from k8s

2021-10-22 Thread dcausse
dcausse closed this task as "Resolved". dcausse claimed this task. dcausse added a comment. updater specific metrics are available here: https://grafana-rw.wikimedia.org/d/fdU5Zx-Mk/wdqs-streaming-updater?orgId=1 flink specific metrics are available here: https://grafana-rw.wikim

[Wikidata-bugs] [Maniphest] T293886: Remove Wikidata query service lag from Wikidata maxlag

2021-10-21 Thread dcausse
dcausse closed this task as "Declined". dcausse added a comment. In T293886#7446988 <https://phabricator.wikimedia.org/T293886#7446988>, @Lydia_Pintscher wrote: > I agree with Lucas. I think we want to have a safeguard in place in case things go wild again and

[Wikidata-bugs] [Maniphest] T293886: Remove Wikidata query service lag from Wikidata maxlag

2021-10-20 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T293886 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Addshore, Lucas_Werkmeister_WMDE, dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen

[Wikidata-bugs] [Maniphest] T293886: Remove Wikidata query service lag from Wikidata maxlag

2021-10-20 Thread dcausse
dcausse added a comment. I don't have much opinion on this so I'll try to ponder to pros & cons: Arguments in favor of removing it: - can be cumbersome to operate (maint script running via a cron on mwmaint1002) - could cause frustration because only well-behaved bots are actu

[Wikidata-bugs] [Maniphest] T285710: WDQS lag detection required manual adjustment during DC switchover

2021-10-20 Thread dcausse
dcausse added a comment. In T285710#7443613 <https://phabricator.wikimedia.org/T285710#7443613>, @Lucas_Werkmeister_WMDE wrote: > Does that mean the periodic `updateQueryServiceLag` should be removed? Currently that’s still running on mwmaint1002. T2217

[Wikidata-bugs] [Maniphest] T293886: Remove Wikidata query service lag from Wikidata maxlag

2021-10-20 Thread dcausse
dcausse created this task. dcausse added projects: Wikidata-Query-Service, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a maintainer of the wikidata software I want to stop propagating the WDQS lag to Wikidata maxlag if it is no longer needed. T221774

[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead

2021-10-20 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T293862 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst

[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead

2021-10-20 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As as a maintainer of a service running on top of the JVM I want the JVM to rapidly quit if it enters a gc death spiral so that the service increase

[Wikidata-bugs] [Maniphest] T285710: WDQS lag detection required manual adjustment during DC switchover

2021-10-19 Thread dcausse
dcausse closed this task as "Resolved". dcausse claimed this task. dcausse added a comment. In T285710#7441835 <https://phabricator.wikimedia.org/T285710#7441835>, @Legoktm wrote: > I think this is resolved now that the streaming updater is in use

[Wikidata-bugs] [Maniphest] T290330: Wikidata Query Service unstable in codfw

2021-10-19 Thread dcausse
dcausse assigned this task to RKemper. dcausse added a comment. @RKemper is there still something left to do related to this? TASK DETAIL https://phabricator.wikimedia.org/T290330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse

[Wikidata-bugs] [Maniphest] T280006: Set up the application authentication for WCQS on commons-query.wikimedia.org

2021-10-19 Thread dcausse
dcausse assigned this task to EBernhardson. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T280006 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL

[Wikidata-bugs] [Maniphest] T290299: Replace token store in MW OAuth WCQS proxy with JWT

2021-10-19 Thread dcausse
dcausse assigned this task to EBernhardson. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. dcausse added a comment. https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/730658 TASK DETAIL https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T292705: Some entities with changed descriptions have old descriptions in WQS and search

2021-10-19 Thread dcausse
dcausse closed this task as a duplicate of T215001: Revisions missing from mediawiki_revision_create. TASK DETAIL https://phabricator.wikimedia.org/T292705 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Gehel, Aklapper

[Wikidata-bugs] [Maniphest] T292705: Some entities with changed descriptions have old descriptions in WQS and search

2021-10-19 Thread dcausse
dcausse added a comment. If this problem happened for both CirrusSearch and WDQS I suspect a problem with changepropagation and/or eventgate. WDQS have been reloaded and no longer shows the affect item. Search does still have the revision 1412772598 <https://www.wikidata.org/w/index.

[Wikidata-bugs] [Maniphest] T291054: wcqs-beta.wmflabs.org SPARQL endpoint is down (500 server error)

2021-10-19 Thread dcausse
dcausse moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. service seems back up TASK DETAIL https://phabricator.wikimedia.org/T291054 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-10-19 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, Zbyszko, RKemper, Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-18 Thread dcausse
dcausse added a subscriber: Cparle. dcausse added a comment. @Cparle I remember you worked on MCR slot filtering on RecentChanges, please let us know if you have suggestions on this approach which share somewhat the same overarching goal. (track changes to file pages with mediainfo data

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-15 Thread dcausse
dcausse added a comment. The two patches are up for discussions and add a new array field (not a big fan of this but could not find a better way to model this) named `rev_slots` with the list of slot record informations: - rev_slot_role - rev_slot_contentmodel - rev_slot_sha1

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-15 Thread dcausse
dcausse claimed this task. dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T293195 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-15 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T293195 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: JAllemandou, Milimetric, Aklapper, Ottomata, Pchelolo, dcausse, Suran38

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2021-10-15 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Addshore, dcausse, Aklapper, Suran38, Invadibot, MPhamWMF, maantietaja, Peteosx1x

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2021-10-15 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Addshore, dcausse, Aklapper, Suran38, Invadibot, MPhamWMF, maantietaja, Peteosx1x

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2021-10-14 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Addshore, dcausse, Aklapper, Suran38, Invadibot, MPhamWMF, maantietaja, Peteosx1x

[Wikidata-bugs] [Maniphest] T241128: EPIC: Reduce the time needed to do the initial WDQS import

2021-10-14 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T241128 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Addshore, dcausse, Aklapper, Suran38, Invadibot, MPhamWMF, maantietaja, Peteosx1x

[Wikidata-bugs] [Maniphest] T262265: Provide real-time updates for WCQS

2021-10-13 Thread dcausse
dcausse added a subtask: T293195: Add MCR slot information to revision-create events. TASK DETAIL https://phabricator.wikimedia.org/T262265 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EBernhardson, dcausse Cc: Back_ache, So9q, Salgo60, Gehel

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-13 Thread dcausse
dcausse added a parent task: T262265: Provide real-time updates for WCQS. TASK DETAIL https://phabricator.wikimedia.org/T293195 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Ottomata, Pchelolo, dcausse, MPhamWMF, CBogen

[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events

2021-10-13 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a consumer of the mediawiki.revision-create events I want to know what slots are available in the revision being created so that I can properly

[Wikidata-bugs] [Maniphest] T284478: wikidata api - wbsearchentities randomly not returning search results

2021-10-12 Thread dcausse
dcausse added a comment. Looking at the codebase I don't understand where this could happen without entering CirrusSearch (unless the `APIAfterExecute` hook is not called on `wbsearchentities` or a bug in the cirrus request logger). Assuming that cirrus was hit I think the problem comes

[Wikidata-bugs] [Maniphest] T293063: Write and adapt Runbooks related to the WDQS Streaming Updater and kubernetes

2021-10-12 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As an SRE operating on the k8s cluster I want to have clear runbooks related to the WDQS Streaming Updater so that I can act on the various

[Wikidata-bugs] [Maniphest] T292404: Move the categories graph out of blazegraph

2021-10-06 Thread dcausse
dcausse added a comment. In T292404#7404318 <https://phabricator.wikimedia.org/T292404#7404318>, @MPhamWMF wrote: > Is this related to T289517 <https://phabricator.wikimedia.org/T289517>? Not directly related as these are two different datasets. This ti

[Wikidata-bugs] [Maniphest] T292404: Move the categories graph out of blazegraph

2021-10-04 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T292404 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314

[Wikidata-bugs] [Maniphest] T292404: Move the categories graph out of blazegraph

2021-10-04 Thread dcausse
dcausse added a project: Wikidata-Query-Service. TASK DETAIL https://phabricator.wikimedia.org/T292404 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-10-01 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-10-01 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-10-01 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-10-01 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T292152: dashboard with daily query service usage not updating

2021-09-30 Thread dcausse
dcausse added a comment. @MPhamWMF see T227782 <https://phabricator.wikimedia.org/T227782>, the data stopped to be officially updated on Aug 2021 (even though something in this data pipeline seemed to have broke earlier around April). TASK DETAIL https://phabricator.wikimedia.org/T

[Wikidata-bugs] [Maniphest] T292073: Investigate the number of queries on DCAT endpoint

2021-09-30 Thread dcausse
dcausse added a comment. It is serving the Data Catalog Vocabulary so it's not used by deepcat. It is just offering sparql endpoint on top of the file https://dumps.wikimedia.org/other/wikibase/wikidatawiki/dcatap.rdf. I think the main benefit of this dataset is the links the dumps but I

[Wikidata-bugs] [Maniphest] T291609: Deleted Wikidata items still returned by WQS

2021-09-27 Thread dcausse
dcausse moved this task from Ready for Development to Needs Reporting on the Discovery-Search (Current work) board. dcausse claimed this task. dcausse added a comment. Ran the script to remove deleted items on the production wdqs machines (these items should disappear from wdqs results

[Wikidata-bugs] [Maniphest] T291488: wdqs1004 lags 16 hours please depool

2021-09-22 Thread dcausse
dcausse assigned this task to Dzahn. dcausse closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T291488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Dzahn, dcausse Cc: Dzahn, RKemper, dcausse, Aklapper, Lydia

[Wikidata-bugs] [Maniphest] T291488: wdqs1004 lags 16 hours please depool

2021-09-21 Thread dcausse
dcausse moved this task from Incoming to Waiting on the Discovery-Search (Current work) board. dcausse added a comment. wdqs1004 is already depooled we just have to wait TASK DETAIL https://phabricator.wikimedia.org/T291488 WORKBOARD https://phabricator.wikimedia.org/project/board/1227

[Wikidata-bugs] [Maniphest] T291488: wdqs1004 lags 16 hours please depool

2021-09-21 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T291488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Dzahn, RKemper, dcausse, Aklapper, Lydia_Pintscher, So9q, MPhamWMF, CBogen

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging

2021-09-21 Thread dcausse
dcausse renamed this task from "wdqs1004 is lagging 5 hours more than all others" to "wdqs1004 is lagging". TASK DETAIL https://phabricator.wikimedia.org/T290832 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q, dcausse

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging 5 hours more than all others

2021-09-21 Thread dcausse
dcausse closed this task as a duplicate of T291488: wdqs1004 lags 16 hours please depool. TASK DETAIL https://phabricator.wikimedia.org/T290832 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q, dcausse Cc: dcausse, RKemper, Dzahn, So9q, Aklapper

[Wikidata-bugs] [Maniphest] T291488: wdqs1004 lags 16 hours please depool

2021-09-21 Thread dcausse
dcausse merged a task: T290832: wdqs1004 is lagging 5 hours more than all others. TASK DETAIL https://phabricator.wikimedia.org/T291488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Dzahn, RKemper, dcausse, Aklapper, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging 5 hours more than all others

2021-09-21 Thread dcausse
dcausse reopened this task as "Open". TASK DETAIL https://phabricator.wikimedia.org/T290832 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q, dcausse Cc: dcausse, RKemper, Dzahn, So9q, Aklapper, Invadibot, MPhamWMF, maantietaj

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging 5 hours more than all others

2021-09-21 Thread dcausse
dcausse closed this task as a duplicate of T291488: wdqs1004 lags 16 hours please depool. TASK DETAIL https://phabricator.wikimedia.org/T290832 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q, dcausse Cc: dcausse, RKemper, Dzahn, So9q, Aklapper

[Wikidata-bugs] [Maniphest] T291488: wdqs1004 lags 16 hours please depool

2021-09-21 Thread dcausse
dcausse merged a task: T290832: wdqs1004 is lagging 5 hours more than all others. dcausse added subscribers: dcausse, RKemper, Dzahn. TASK DETAIL https://phabricator.wikimedia.org/T291488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging 5 hours more than all others

2021-09-21 Thread dcausse
dcausse moved this task from Incoming to Waiting on the Discovery-Search (Current work) board. dcausse added a comment. Thanks @So9q for the report and @Dzahn for the depool! We'll repool once the lag is back to normal. TASK DETAIL https://phabricator.wikimedia.org/T290832 WORKBOARD

[Wikidata-bugs] [Maniphest] T290832: wdqs1004 is lagging 5 hours more than all others

2021-09-21 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T290832 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q, dcausse Cc: dcausse, RKemper, Dzahn, So9q, Aklapper, Invadibot, MPhamWMF

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-09-20 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-09-17 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application

2021-09-17 Thread dcausse
dcausse closed subtask T286890: Checkpoint _metadata has grown up to 70Mb as Declined. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Mohammed_Sadat_WMDE, So9q, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] T286890: Checkpoint _metadata has grown up to 70Mb

2021-09-17 Thread dcausse
dcausse moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. dcausse closed this task as "Declined". dcausse added a comment. Analyzed the large _metadata file and it has 3 operators with very large states esp. `max-part-coun

[Wikidata-bugs] [Maniphest] T286890: Checkpoint _metadata has grown up to 70Mb

2021-09-16 Thread dcausse
dcausse claimed this task. dcausse moved this task from Ready for Development to Incoming on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T286890 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-09-15 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production

2021-09-15 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T288231 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264

[Wikidata-bugs] [Maniphest] T288230: Promote MediaInfo RDF format to stable

2021-09-13 Thread dcausse
dcausse added a comment. @CBogen I would not block WCQS work because of this, esp. because we agree on the substance that this format must be stable. I would perhaps clarify quickly (before going live) what the user documentation for the RDF format will look like, esp. because MediaInfo

[Wikidata-bugs] [Maniphest] T284446: Raise alert when Blazegraph journal grows faster than expected

2021-09-13 Thread dcausse
dcausse claimed this task. dcausse moved this task from Ready for Development to Needs review on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T284446 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T276467: Ensure we have proper monitoring / alerting on the new Flink based WDQS Streaming Updater

2021-09-08 Thread dcausse
dcausse claimed this task. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T276467 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T283591: StateExtractionJob is too slow

2021-09-08 Thread dcausse
dcausse moved this task from In Progress to Waiting on the Discovery-Search (Current work) board. dcausse added a comment. Posted a message to the flink ml asking for help https://lists.apache.org/thread.html/rb8377eb10f0b1736264ae3dbb84986a5ff5907ec06431e25bff4dcda%40

[Wikidata-bugs] [Maniphest] T290545: Wikidata Query Service inaccessible via GUI, only partially accessible via embedded queries

2021-09-08 Thread dcausse
dcausse moved this task from Incoming to Needs Reporting on the Discovery-Search (Current work) board. dcausse assigned this task to RKemper. dcausse closed this task as "Resolved". dcausse added a comment. Closing, the UI should be accessible again. TASK DETA

[Wikidata-bugs] [Maniphest] T290545: Wikidata Query Service inaccessible via GUI, only partially accessible via embedded queries

2021-09-08 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T290545 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gnoeee, Fuzheado, Aklapper, Daniel_Mietchen, MPhamWMF, CBogen, Namenlos314

[Wikidata-bugs] [Maniphest] T289836: Upgrade to latest flink (1.14)

2021-09-07 Thread dcausse
dcausse renamed this task from "Upgrade to latest flink (1.13.2)" to "Upgrade to latest flink (1.14)". dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289836 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/pa

[Wikidata-bugs] [Maniphest] T288230: Promote MediaInfo RDF format to stable

2021-09-06 Thread dcausse
dcausse added a comment. In T288230#7333958 <https://phabricator.wikimedia.org/T288230#7333958>, @Cparle wrote: > 1. there is no team to oversee its stability (we're the structured data team, not the commons team) I think that the team owning WikibaseMediaInfo should be

[Wikidata-bugs] [Maniphest] T283591: StateExtractionJob is too slow

2021-09-06 Thread dcausse
dcausse claimed this task. dcausse moved this task from incoming to in progress on the Wikidata board. TASK DETAIL https://phabricator.wikimedia.org/T283591 WORKBOARD https://phabricator.wikimedia.org/project/board/71/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] T284137: Allow federated queries with the Lingua Libre SPARQL endpoint

2021-09-06 Thread dcausse
dcausse added a comment. @WikiLucas00 my apologies, I completely missed your ping, yes lingualibre can be queried directly from `query.wikidata.org`. Regarding your second question, sadly it is unlikely that the performances of a single query will be better once wcqs is running

<    1   2   3   4   5   6   7   8   9   10   >