[Wikidata-bugs] [Maniphest] [Updated] T222404: Tasks requiring Blazegraph reload

2019-05-02 Thread Smalyshev
Smalyshev added projects: Wikidata-Query-Service, Tracking-Neverending. Restricted Application added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T222404 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper

[Wikidata-bugs] [Maniphest] [Edited] T218522: SPARQL incorrectly changes URI's from http to https

2019-05-02 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T218522 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Base, Aklapper, Lydia_Pintscher, Smalyshev, Multichill, alaa_wmde, Nandana, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Closed] T221146: WDQS confuses strings with and without U+00AD

2019-05-02 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev added a comment. This is fixed now. TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Liridon, Bdijkstra

[Wikidata-bugs] [Maniphest] [Commented On] T221921: Provision search endpoint for SDC. Requirements from Product Team.

2019-05-02 Thread Smalyshev
Smalyshev added a comment. The outcomes of the chat today seem to be: 1. We're making separate Commons query service, on separate servers 2. The estimate for the data size is: the number of items will eventually be the same as Wikidata, but the item size probably substantially less

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-02 Thread Smalyshev
Smalyshev added a comment. BTW I am not sure how MediaInfoIdLookup is supposed to work: if ( !$title->inNamespace( $this->mediaInfoNamespace ) ) { return null; } try { return new Media

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-02 Thread Smalyshev
Smalyshev added a comment. > But the question remains - how would dumpRdf know to use it? Exactly. The question is not how to get MediaInfoId from page id - it's easy. The question is how dumpRdf would even know mediainfo is there on the page and know that it needs to use special c

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-02 Thread Smalyshev
Smalyshev added a comment. @Cparle Yes, I know, but this only works for MediaInfo. dumpRdf code does not know that particular page contains a MediaInfo slot (or that such thing as MediaInfo exists at all). That's the whole problem. TASK DETAIL https://phabricator.wikimedia.org/T99

[Wikidata-bugs] [Maniphest] [Merged] T221916: Create RDF export for structured data stored for files

2019-05-02 Thread Smalyshev
Smalyshev merged a task: T215341: RDF for MediaInfo entities. Smalyshev added a subscriber: WMDE-leszek. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: WMDE-leszek, Poyekhali

[Wikidata-bugs] [Maniphest] [Updated] T215341: RDF for MediaInfo entities

2019-05-02 Thread Smalyshev
Smalyshev closed this task as a duplicate of T221916: Create RDF export for structured data stored for files. TASK DETAIL https://phabricator.wikimedia.org/T215341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, WMDE-leszek

[Wikidata-bugs] [Maniphest] [Updated] T215341: RDF for MediaInfo entities

2019-05-02 Thread Smalyshev
Smalyshev closed this task as a duplicate of T221917: Create RDF dump of structured data on Commons. TASK DETAIL https://phabricator.wikimedia.org/T215341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, WMDE-leszek

[Wikidata-bugs] [Maniphest] [Merged] T221917: Create RDF dump of structured data on Commons

2019-05-02 Thread Smalyshev
Smalyshev merged a task: T215341: RDF for MediaInfo entities. TASK DETAIL https://phabricator.wikimedia.org/T221917 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: WMDE-leszek, Poyekhali, Steinsplitter, Aklapper, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] [Updated] T221146: WDQS confuses strings with and without U+00AD

2019-05-02 Thread Smalyshev
Smalyshev added a project: User-Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Liridon, Bdijkstra, Aklapper, alaa_wmde, Nandana, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Triaged] T194144: Find a solution for SpecialEntitiesWithoutPage (EntitiesWithoutTermFinder)

2019-05-02 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T194144 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Addshore, Aklapper, EBernhardson, Smalyshev, Lydia_Pintscher, hoo,

[Wikidata-bugs] [Maniphest] [Triaged] T187314: Display labels for entity values at result visualisation step in Wikibase Query Engine

2019-05-02 Thread Smalyshev
Smalyshev triaged this task as "Low" priority. TASK DETAIL https://phabricator.wikimedia.org/T187314 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Manu1400, Smalyshev, Jonas, Lydia_Pintscher, Lucas_Werkmeister_WMDE, Aklapp

[Wikidata-bugs] [Maniphest] [Triaged] T222302: Too many RDF label forms for MediaInfo

2019-05-02 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222302 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: WMDE-leszek, Lucas_Werkmeister_WMDE, Tpt, Addshore, Ramsey-WMF, Aklapper,

[Wikidata-bugs] [Maniphest] [Triaged] T222321: Make /entity/ alias work for Commons

2019-05-02 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222321 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Tpt, Addshore, Ramsey-WMF, Lydia_Pintscher,

[Wikidata-bugs] [Maniphest] [Commented On] T214896: Error when exporting entity on federated-commons test system to RDF

2019-05-01 Thread Smalyshev
Smalyshev added a comment. Is this issue still relevant? TASK DETAIL https://phabricator.wikimedia.org/T214896 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, WMDE-leszek, Lucas_Werkmeister_WMDE, Aklapper, alaa_wmde

[Wikidata-bugs] [Maniphest] [Created] T222321: Make /entity/ alias work for Commons

2019-05-01 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo, User-Smalyshev. TASK DESCRIPTION Wikidata has entity URLs <https://meta.wikimedia.org/wiki/Wikidata/Notes/URI_scheme> in the form of http://www.wikida

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev added a comment. I think `EntityByLinkedTitleLookup` can be used for this, with slight modification that enables `SiteLinkTable` to produce links for local Wikidata without going to the database. Also, `SiteLinkTable` for some reason only produces item IDs. Maybe that should

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev added a comment. Looking further into this, we can check if the page has the slot with this: 1. Create WikiPage from page ID/title 2. $revisionRecord = $page->getRevisionRecord() 3. $mediaInfoSlot = $revisionRecord ->getSlot( MediaInfo::ENTITY_TYPE ) But it is

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev added a comment. EntityIdLookup has: public function getEntityIdForTitle( Title $title ); but I am not sure how it's supposed to work since it does not have slot ID or anything like that as a parameter... And the main implementation does

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev added a comment. MediaInfo sets namespace as: public static function onWikibaseRepoEntityNamespaces( &$entityNamespacesSetting ) { // Tell Wikibase where to put our entity content. $entityNamespacesSetting[ MediaInfo::ENTITY_

[Wikidata-bugs] [Maniphest] [Commented On] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev added a comment. Looks like `mediainfo` type is mapped to namespace 6, but the real namespace is 144, where we get entities with right titles. Maybe config problem? TASK DETAIL https://phabricator.wikimedia.org/T99 EMAIL PREFERENCES https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Triaged] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T99 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Cparle, daniel, Lucas_Werkmeister_WMDE, Tpt, Addshore, Ramsey-WMF,

[Wikidata-bugs] [Maniphest] [Triaged] T221917: Create RDF dump of structured data on Commons

2019-05-01 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T221917 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper, Lydia_Pintscher, Bugreporter, Tgr,

[Wikidata-bugs] [Maniphest] [Triaged] T222306: RDF export generates wrong IDs for federated entities

2019-05-01 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222306 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: WMDE-leszek, Lucas_Werkmeister_WMDE, Addshore, Ramsey-WMF, Lydia

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T222306: RDF export generates wrong IDs for federated entities

2019-05-01 Thread Smalyshev
Smalyshev added a subscriber: WMDE-leszek. Smalyshev added a comment. Looking at the code, the ID is plain `P64`, even though it is coming from different repository, but the ID returns empty string as repository name. So I am confused about how exactly this is supposed to work

[Wikidata-bugs] [Maniphest] [Unassigned] T222306: RDF export generates wrong IDs for federated entities

2019-05-01 Thread Smalyshev
Smalyshev removed Smalyshev as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T222306 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Ramsey-WMF, Lydia_Pintscher, Aklapper

[Wikidata-bugs] [Maniphest] [Unassigned] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev removed Smalyshev as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T99 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Cparle, daniel, Lucas_Werkmeister_WMDE, Tpt, Addshore, Ramsey-WMF, Aklapper

[Wikidata-bugs] [Maniphest] [Unassigned] T222302: Too many RDF label forms for MediaInfo

2019-05-01 Thread Smalyshev
Smalyshev removed Smalyshev as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T222302 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: WMDE-leszek, Lucas_Werkmeister_WMDE, Tpt, Addshore, Ramsey-WMF, Aklapper

[Wikidata-bugs] [Maniphest] [Edited] T222306: RDF export generates wrong IDs for federated entities

2019-05-01 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T222306 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Ramsey-WMF, Lydia_Pintscher, Aklapper, Smalyshev, alaa_wmde

[Wikidata-bugs] [Maniphest] [Created] T222306: RDF export generates wrong IDs for federated entities

2019-05-01 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo, User-Smalyshev. TASK DESCRIPTION When generating RDF for MediaInfo entity, I get this: wd:M10659 a wikibase:Mediainfo, wikibase:MediaInfo

[Wikidata-bugs] [Maniphest] [Created] T222302: Too many RDF label forms for MediaInfo

2019-05-01 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo, User-Smalyshev. TASK DESCRIPTION Right now the label forms produced for MediaInfo (e.g. see https://federated-commons.wmflabs.org/wiki/Special:EntityData/M10659

[Wikidata-bugs] [Maniphest] [Created] T222299: dumpRdf does not work with MediaInfo entities

2019-05-01 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo, User-Smalyshev. TASK DESCRIPTION dumpRdf uses SqlEntityIdPager to fetch IDs by namespace linked to the entity type, and then does this: $position = (int

[Wikidata-bugs] [Maniphest] [Claimed] T221916: Create RDF export for structured data stored for files

2019-05-01 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper, Lydia_Pintscher, Tgr, Ramsey-WMF, Jarekt, Addshore, Tpt, MB-one

[Wikidata-bugs] [Maniphest] [Triaged] T221916: Create RDF export for structured data stored for files

2019-05-01 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board. TASK DETAIL https://phabricator.wikimedia.org/T221916 WORKBOARD https://phabricator.wikimedia.org/project/board/2921/ EMAIL PREFERENC

[Wikidata-bugs] [Maniphest] [Updated] T213375: Inline value and reference URIs

2019-04-30 Thread Smalyshev
Smalyshev added a comment. Queries for tests: P8462 (An Untitled Masterwork) <https://phabricator.wikimedia.org/P8462> I will probably add more but these should be good for now. TASK DETAIL https://phabricator.wikimedia.org/T213375 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Triaged] T222181: “Edit visually” broken on embed.htm

2019-04-30 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222181 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Smalyshev Cc: Aklapper, Lucas_Werkmeister_WMDE, alaa_wmde, j

[Wikidata-bugs] [Maniphest] [Triaged] T222133: "Edit SPARQL" link is broken in embed.html

2019-04-30 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Smalyshev Cc: Lucas_Werkmeister_WMDE, Aklapper, Yair_rand,

[Wikidata-bugs] [Maniphest] [Triaged] T222182: Short URL doesn’t work on embed.html

2019-04-30 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T222182 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Smalyshev Cc: Lucas_Werkmeister_WMDE, Aklapper, alaa_wmde, j

[Wikidata-bugs] [Maniphest] [Changed Project Column] T55652: Special:Search doesn't use labels and descriptions for suggestions but just the item ID

2019-04-30 Thread Smalyshev
Smalyshev moved this task from Wikidata Search to Current work on the Discovery-Search board. Smalyshev edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T55652 WORKBOARD https://phabricator.wikimedia.org/project

[Wikidata-bugs] [Maniphest] [Created] T222098: Replace wdqs1003

2019-04-29 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION wdqs1003 is one of the older servers on our cluster, and seems to get performance issues much

[Wikidata-bugs] [Maniphest] [Updated] T221916: Create RDF export for structured data stored for files

2019-04-29 Thread Smalyshev
Smalyshev added a project: User-Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper, Lydia_Pintscher, Tgr, Ramsey-WMF, Jarekt, Addshore

[Wikidata-bugs] [Maniphest] [Updated] T187314: Display labels for entity values at result visualisation step in Wikibase Query Engine

2019-04-29 Thread Smalyshev
Smalyshev added a project: patch-welcome. TASK DETAIL https://phabricator.wikimedia.org/T187314 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Manu1400, Smalyshev, Jonas, Lydia_Pintscher, Lucas_Werkmeister_WMDE, Aklapper, TomT0m

[Wikidata-bugs] [Maniphest] [Edited] T221916: Create RDF export for structured data stored for files

2019-04-28 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper, Lydia_Pintscher, Tgr, Ramsey-WMF, Jarekt, Addshore, Tpt

[Wikidata-bugs] [Maniphest] [Created] T222009: Vagrant mediainfo role incompatible with wikidata

2019-04-26 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: MediaWiki-Vagrant, WikibaseMediaInfo, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION If I try to enable both wikidata and mediainfo roles, I get this: ==> default: Error: Evaluation Error: Error wh

[Wikidata-bugs] [Maniphest] [Claimed] T221146: WDQS confuses strings with and without U+00AD

2019-04-26 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Liridon, Bdijkstra, Aklapper, alaa_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] [Updated] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-26 Thread Smalyshev
Smalyshev removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Smalyshev, BBlack, Aklapper, Gehel, alaa_wmde

[Wikidata-bugs] [Maniphest] [Closed] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-26 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev added a comment. Looks like it's not broken anymore. Thanks! TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: noarave, Sm

[Wikidata-bugs] [Maniphest] [Updated] T221916: Create RDF export for structured data stored for files

2019-04-25 Thread Smalyshev
Smalyshev added a parent task: T221917: Create RDF dump of structured data on Commons. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T221917: Create RDF dump of structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev added a subtask: T221916: Create RDF export for structured data stored for files. TASK DETAIL https://phabricator.wikimedia.org/T221917 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper

[Wikidata-bugs] [Maniphest] [Created] T221917: Create RDF dump of structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo. TASK DESCRIPTION We should set up a regular dump of all structured data entities on Commons, akin to dumps of Wikidata entities we have now. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Created] T221916: Create RDF export for structured data stored for files

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo. TASK DESCRIPTION Structured data information on Commons should have RDF representation, just like the information on Wikidata has. We need to create code

[Wikidata-bugs] [Maniphest] [Updated] T141602: [Story] Provide a SPARQL query service for structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev edited projects, added Wikidata-Query-Service, Epic; removed Story. TASK DETAIL https://phabricator.wikimedia.org/T141602 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Salgo60, MB-one, Tpt, Addshore

[Wikidata-bugs] [Maniphest] [Created] T221882: WDQS public cluster capacity upgrade

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Discovery. Restricted Application added a project: Wikidata. TASK DESCRIPTION As can be witnessed from the WDQS lag graph <https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1=now-90d=now=8>

[Wikidata-bugs] [Maniphest] [Commented On] T221632: Storage capacity upgrade for WDQS

2019-04-25 Thread Smalyshev
Smalyshev added a comment. > I don't think it makes sense to perpetuate a vertical scaling model With regard to disk space, I don't think we have a choice at least for the next FY. Even if we find alternative backend that can be a drop-in replacement for Blazegraph (or some

[Wikidata-bugs] [Maniphest] [Claimed] T212550: Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata

2019-04-24 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Krinkle, Ottomata, Aklapper, Smalyshev, aaron, alaa_wmde, joker88john, CucyNoiD, Nandana, NebulousIris

[Wikidata-bugs] [Maniphest] [Claimed] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-24 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Smalyshev, BBlack, Aklapper, Gehel, alaa_wmde, joker88john

[Wikidata-bugs] [Maniphest] [Retitled] T212550: Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata

2019-04-24 Thread Smalyshev
Smalyshev renamed this task from "Implement support for ChronologyProtection in RDF export" to "Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata". TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL

[Wikidata-bugs] [Maniphest] [Closed] T221407: Unrecognized subject messages in Updater

2019-04-23 Thread Smalyshev
Smalyshev closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Gehel, Addshore, WMDE-leszek, Lucas_Werkmeister_WMDE, Aklapper, Smalyshev,

[Wikidata-bugs] [Maniphest] [Claimed] T221407: Unrecognized subject messages in Updater

2019-04-23 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Gehel, Addshore, WMDE-leszek, Lucas_Werkmeister_WMDE, Aklapper, Smalyshev, alaa_wmde, joker88john

[Wikidata-bugs] [Maniphest] [Created] T221709: scap service restarts for WDQS are inconsistent

2019-04-23 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Scap. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Right now, when WDQS is deployed, only service `wdqs-blazegraph` is restarted

[Wikidata-bugs] [Maniphest] [Edited] T221632: Storage capacity upgrade for WDQS

2019-04-23 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T221632 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, Smalyshev Cc: Smalyshev, Aklapper, Gehel, alaa_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] [Commented On] T221632: Storage capacity upgrade for WDQS

2019-04-23 Thread Smalyshev
Smalyshev added a comment. This also makes me note that if we do introduce sharding (in any shape, either with Blazegraph or another solution) we'd need even more servers, since each shard would need to be at least on 2-3 servers to survive, so for sharding to make any sense we'd need

[Wikidata-bugs] [Maniphest] [Closed] T221405: DCATAP dump reload does not delete old namespace

2019-04-23 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221405 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Smalyshev, alaa_wmde, joker88john, CucyNoi

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-21 Thread Smalyshev
Smalyshev added a comment. Deploying the constraints patch on test servers seems to eliminate these messages. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Addshore, WMDE

[Wikidata-bugs] [Maniphest] [Triaged] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. Smalyshev added a project: Discovery-Wikidata-Query-Service-Sprint. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Adds

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added subscribers: Lucas_Werkmeister_WMDE, WMDE-leszek, Addshore. Smalyshev added a comment. I think I know the reason... Constraint violations are always fetched for latest version, while revision fetches may be fetched for non-current ones. This means that constraint violation

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Results of caching can be seen here: https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1=now-30d=now=24_name=wdqs-internal Deploy date is Apr 11, fetch time drops from 135/195 ms (eqiad/codfw) to 90/150 ms when requests are cached. TASK

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Also, errors on different servers do not seem to match, even though the content is supposed to be exactly the same - this is the whole point of caching after all! Something weird is definitely going on. TASK DETAIL https://phabricator.wikimedia.org/T221407

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. First error on wdq24 is from 2019-04-11 07:45:56.038. Patch was merged also on Apr 11 and Updater was restarted with it on 07:42:21.097. I'm pretty much sure it's connected - still not sure how. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Curiously, this happens only on hosts where revision-fetch is enabled. I wonder whether it's related, though I am not sure how. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-18 Thread Smalyshev
Smalyshev added a comment. @aaron Is there any docs how to use the client ID on the client side? I see there's support for `cpPosIndex` cookie which is `$index@$time#$clientId` but if I have only client ID that is not going to help me. So I am not sure how would one use it - I can't find

[Wikidata-bugs] [Maniphest] [Retitled] T221146: WDQS confuses strings with and without U+00AD

2019-04-18 Thread Smalyshev
Smalyshev renamed this task from "WDQS returns stale data" to "WDQS confuses strings with and without U+00AD". Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricato

[Wikidata-bugs] [Maniphest] [Commented On] T221146: WDQS confuses strings with and without U+00AD

2019-04-18 Thread Smalyshev
Smalyshev added a comment. See also https://github.com/blazegraph/database/issues/93 TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Liridon, Bdijkstra, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T221146: WDQS returns stale data

2019-04-18 Thread Smalyshev
Smalyshev added a comment. The data seems to be updated in the database (revision 915966700). I suspect it may be the same issue as with T197447 <https://phabricator.wikimedia.org/T197447> - Blazegraph conflates strings with some invisible characters and strings without them. TASK

[Wikidata-bugs] [Maniphest] [Created] T221407: Unrecognized subject messages in Updater

2019-04-18 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Updater logs show these messages: 05:26:33.222 [update 5] INFO

[Wikidata-bugs] [Maniphest] [Created] T221405: DCATAP dump reload does not delete old namespace

2019-04-18 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When we load new DCATAP dump into new namespace, old namespace is supposed to be deleted

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-18 Thread Smalyshev
Smalyshev added a comment. This is probably the request for @aaron, he knows the most about it. TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Ottomata, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Updated] T217925: Keep global "last seen revision" map for Updater

2019-04-17 Thread Smalyshev
Smalyshev removed a project: Discovery-Wikidata-Query-Service-Sprint. TASK DETAIL https://phabricator.wikimedia.org/T217925 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Gehel, Aklapper, Smalyshev, alaa_wmde

[Wikidata-bugs] [Maniphest] [Commented On] T221315: Determine future of bare-metal hosting for services like WDQS, ElasticSearch, RESTBase Cassandra, etc.

2019-04-17 Thread Smalyshev
Smalyshev added a comment. The web UI frontend is just a bunch of HTML files, so there's not much to move. There's however nginx frontend, which serves in dual capacity as both gateway to database backend and UI server. It is possible to run either function on a separate container

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-17 Thread Smalyshev
Smalyshev added a comment. Still getting failures, see: https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-java8-docker/855/console TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Created] T221127: URL shortener widget is hard to copy

2019-04-16 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata Query UI. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION There are reports that URL shortener widget is hard to work with: @Ladsgroup: > whe

[Wikidata-bugs] [Maniphest] [Created] T221126: Improve error reporting on URL shortener

2019-04-16 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata Query UI. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Right now, the only error URL shortener produces is "URL shortening failed". We proba

[Wikidata-bugs] [Maniphest] [Commented On] T112715: Enable different URL shorteners for WDQS

2019-04-16 Thread Smalyshev
Smalyshev added a comment. > So there is no difference between "failed because URL is too long" and "failed because so many people use the service that it's been blocked by rate limit" Yes, error reporting right now is most rudimentary. We probably should fix tha

[Wikidata-bugs] [Maniphest] [Edited] T190422: On WDQS some messages are not translated when switching languages

2019-04-16 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T190422 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Micru, alaa_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-15 Thread Smalyshev
Smalyshev added a comment. @aaron I have started looking into it, and I am not sure how one can get an instance of ChronologyProtector in the code that needs it. The only place I can see that uses ChronologyProtector is LBFactory class, but getChronologyProtector() is protected there, so

[Wikidata-bugs] [Maniphest] [Commented On] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev added a comment. @Ladsgroup which one? With the query (top left) or the results (down right in the menu)? In any case, probably makes sense to open new task with a screen shot. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Unblock] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed subtask T218568: Allow CORS from query.wikidata.org to production wikis as Declined. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing

[Wikidata-bugs] [Maniphest] [Declined] T218568: Allow CORS from query.wikidata.org to production wikis

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Declined". Smalyshev added a comment. Looks like URL shortener works without it, so we don't need this for now. TASK DETAIL https://phabricator.wikimedia.org/T218568 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailp

[Wikidata-bugs] [Maniphest] [Unblock] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed subtask T219968: Enable the URLshortener for WDQS as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing, Lucas_Werkmeister_WMDE, Ladsgroup

[Wikidata-bugs] [Maniphest] [Closed] T219968: Enable the URLshortener for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T219968 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Salgo60, CennoxX, Sylvain_WM

[Wikidata-bugs] [Maniphest] [Closed] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-10 Thread Smalyshev
Smalyshev added a comment. The latest patch doesn't seem to help, because test is run by pom.xml as it seems. Please see full log in https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-java8-docker/850/console TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T217557: Socket timeout on wdqs.svc.eqiad.wmnet

2019-04-10 Thread Smalyshev
Smalyshev added a comment. @Gehel any input on this? TASK DETAIL https://phabricator.wikimedia.org/T217557 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Mholloway, Aklapper, Mathew.onipe, Smalyshev, Gehel, alaa_wmde

[Wikidata-bugs] [Maniphest] [Edited] T220666: quibble-vendor-mysql-hhvm-docker fails in Wikibase

2019-04-10 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T220666 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, alaa_wmde, Nandana, Mringgaard, Lahi, Gq86, GoranSMilovanovic, QZanden

[Wikidata-bugs] [Maniphest] [Created] T220666: quibble-vendor-mysql-hhvm-docker fails in Wikibase

2019-04-10 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Campsite, Lexicographical data. Restricted Application added a project: Wikidata. TASK DESCRIPTION Getting this failure on Wikibase test: 19:00:25 1) Special:RecentChanges shows lemmas in title links to lexemes: 19:00:25

[Wikidata-bugs] [Maniphest] [Commented On] T220046: Set up developer account for Igor Kim

2019-04-09 Thread Smalyshev
Smalyshev added a comment. @thcipriani @mmodell is this something you would be able to do - create the Gerrit account for Igor manually? TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T200612: Wikidata's SPARQL endpoint doesn't escape commas in IRIs in CSV output

2019-04-09 Thread Smalyshev
Smalyshev added a comment. Turns out Sesame 2.8 has pretty big difference from 2.7 - in RDF 1.1/SPARQL 1.1 there are no "simple literals" anymore, i.e. literal `"abc"` is the same as literal `"abc"^^xsd:string` and the only type of plain literal is languag

[Wikidata-bugs] [Maniphest] [Edited] T218716: Remove Elastic/Cirrus code from Wikibase repo

2019-04-09 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T218716 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Lydia_Pintscher, WMDE-leszek, RazShuty, thiemowmde, Aleksey_WMDE, Addshore

<    1   2   3   4   5   6   7   8   9   10   >