[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

2019-02-13 Thread Smalyshev
Smalyshev added a comment. Thanks, T213318 makes it a bit clearer though not entirely 100% clear which parts stay in PHP and which parts move to JS. Would it also be true that there is no way to render Wikibase content (even without editing) on non-_javascript_ browsers? The SPA approach

[Wikidata-bugs] [Maniphest] [Commented On] T213318: Wikibase Front-End Architecture

2019-02-13 Thread Smalyshev
Smalyshev added a comment. A useful Wikibase (with query service etc) can already not simply be installed on a LAMP stack. Well, obviously you need Java for running WDQS, but otherwise it can be pretty easily installed on any common LAMP machine. Of course, assembling all configuration

[Wikidata-bugs] [Maniphest] [Triaged] T215980: Move LDFServer fork repo to gerrit

2019-02-12 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority.Smalyshev added a comment. I've requested the gettit repo and as soon as it's made, I'll move the fork there.TASK DETAILhttps://phabricator.wikimedia.org/T215980EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailp

[Wikidata-bugs] [Maniphest] [Created] T215980: Move LDFServer fork repo to gerrit

2019-02-12 Thread Smalyshev
Smalyshev created this task.Smalyshev added a project: Wikidata-Query-Service.Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata. TASK DESCRIPTIONWe're using slightly modified version of LDF Server for Blazegraph in WDQS. Right now it is hosted

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

2019-02-11 Thread Smalyshev
Smalyshev added a comment. I get the idea of server-side HTML rendering to avoid delays. But I am kinda questioning whether the advantage of splitting code outside of PHP into separate service and all the complexities that follow from that. Maybe there's an easier way to achieve the same benefit

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2019-02-11 Thread Smalyshev
Smalyshev added a comment. Yes, judging from our preliminary test, if we get uncontested use of the server or even a certain chunk of it maybe (not sure if possible?) it would be enough. Note that an interesting scenario that we want to test in foreseeable future involves cluster setup, so we'd

[Wikidata-bugs] [Maniphest] [Unassigned] T189744: Add hints parameter to wbsearchentities

2019-02-11 Thread Smalyshev
Smalyshev removed Smalyshev as the assignee of this task. TASK DETAILhttps://phabricator.wikimedia.org/T189744EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Nuria, Jonas, EBernhardson, gerritbot, Lydia_Pintscher, daniel, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Updated] T215684: Deploy & test WikibaseCirrusSearch on beta cluster

2019-02-11 Thread Smalyshev
Smalyshev edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAILhttps://phabricator.wikimedia.org/T215684EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Aklapper, Smalyshev, ET4Eva, CucyNoiD

[Wikidata-bugs] [Maniphest] [Created] T215751: Make vagrant support WikibaseCirrusSearch

2019-02-10 Thread Smalyshev
Smalyshev created this task.Smalyshev triaged this task as "Normal" priority.Smalyshev added projects: Wikidata, User-Smalyshev, Discovery, Discovery-Search (Current work), MediaWiki-Vagrant.Restricted Application added a subscriber: Aklapper. TASK DESCRIPTIONMake Vagrant recipe tha

[Wikidata-bugs] [Maniphest] [Created] T215684: Deploy & test WikibaseCirrusSearch on beta cluster

2019-02-08 Thread Smalyshev
Smalyshev created this task.Smalyshev triaged this task as "Normal" priority.Smalyshev added projects: Wikidata, User-Smalyshev, Discovery, CirrusSearch, Discovery-Search (Current work).Restricted Application added a subscriber: Aklapper.Restricted Application edited projects, added

[Wikidata-bugs] [Maniphest] [Created] T215556: Enable phan checks for WikibaseLexeme extension

2019-02-07 Thread Smalyshev
Smalyshev created this task.Smalyshev added projects: Lexicographical data, Continuous-Integration-Config.Restricted Application added a project: Wikidata. TASK DESCRIPTIONCurrently, WikibaseLexeme extension does not seem to have phan checks enabled. This means, among other things, that if its

[Wikidata-bugs] [Maniphest] [Closed] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T215503EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: EBernhardson, dcausse, gerritbot, TerraCodes, Liuxinyu970226,

[Wikidata-bugs] [Maniphest] [Commented On] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev added a comment. Also, we probably need some test to capture this situation in the future. Looks like current tests do not cover this branch.TASK DETAILhttps://phabricator.wikimedia.org/T215503EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Updated] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev added a parent task: T206670: 1.33.0-wmf.16 deployment blockers. TASK DETAILhttps://phabricator.wikimedia.org/T215503EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: TerraCodes, Liuxinyu970226, Smalyshev, Lea_Lacroix_WMDE

[Wikidata-bugs] [Maniphest] [Triaged] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev removed a project: Release-Engineering-Team.Smalyshev triaged this task as "Unbreak Now!" priority.Smalyshev added a comment.Restricted Application added subscribers: Liuxinyu970226, TerraCodes. Ah, I see what's up - Wikibase has been updated, but WikibaseLexeme was not.TASK D

[Wikidata-bugs] [Maniphest] [Updated] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev added a project: Wikimedia-production-error. TASK DETAILhttps://phabricator.wikimedia.org/T215503EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Lea_Lacroix_WMDE, Lexicographical data, KaMan, Nandana, NebulousIris, Mringgaard

[Wikidata-bugs] [Maniphest] [Updated] T215503: PHP fatal error: Class undefined: CirrusSearch\Search\EmptyResultSet

2019-02-07 Thread Smalyshev
Smalyshev added a project: Release-Engineering-Team.Smalyshev added a comment. Looks like some deployment issue - 1.33.0-wmf.14 has this class, but 1.33.0-wmf.16 does not.TASK DETAILhttps://phabricator.wikimedia.org/T215503EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Closed] T160259: [Story] RDF for Lexemes, Forms and Senses

2019-02-05 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T160259EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Lydia_Pintscher, Fnielsen, ArthurPSmith, Tpt,

[Wikidata-bugs] [Maniphest] [Unblock] T193645: [Epic] querying for lexicographical data

2019-02-05 Thread Smalyshev
Smalyshev closed subtask T160259: [Story] RDF for Lexemes, Forms and Senses as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T193645EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: johl, Yurik, Lea_Lacroix_WMDE, Smalyshev

[Wikidata-bugs] [Maniphest] [Commented On] T165473: [Epic] new visualizations for Wikidata Query Service

2019-02-04 Thread Smalyshev
Smalyshev added a comment. There's also this library I've encountered recently: https://cytoscape.org/ may be interesting for graph visualizations.TASK DETAILhttps://phabricator.wikimedia.org/T165473EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc

[Wikidata-bugs] [Maniphest] [Commented On] T204813: Allow looking for items and lexemes namespaces together by default

2019-02-02 Thread Smalyshev
Smalyshev added a comment. Would it be acceptable to show the results for Item namespace and Lexeme namespace separately - e.g. in separate tabs? If not, what would be the expected UI - would both kinds of results displayed together with distinction only by prefix or with some separation between

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T194968: Enable search in all wikidata namespaces combined

2019-02-02 Thread Smalyshev
Smalyshev added a subscriber: dcausse.Smalyshev added a comment. After discussing this on the offsite, the current approach seems to be like this: For full-text search with several kinds of namespaces, it's probably best to separate the searches and display each kind in the separate tab (we'd

[Wikidata-bugs] [Maniphest] [Updated] T215073: OSM map layers other than the default should be displayable in the Wikidata Query Service

2019-02-01 Thread Smalyshev
Smalyshev edited projects, added Wikidata Query UI; removed Wikidata-Query-Service.Smalyshev added a comment. We do not have editorial control over the data in Wikidata, yet less on maps provided by the third party. We also are not subject neither to Indian law nor the Twitter audience &quo

[Wikidata-bugs] [Maniphest] [Triaged] T214032: query service should send a retry-after header with a 429 response

2019-01-31 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAILhttps://phabricator.wikimedia.org/T214032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Lucas_Werkmeister_WMDE, Smalyshev, Aklapper, Addshore, CucyNoi

[Wikidata-bugs] [Maniphest] [Updated] T209621: Confirm perceived slowness in job queue for getting new captions available for searching in Commons

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added SDC General, Multimedia, Discovery-Search; removed Discovery-Search (Current work).Restricted Application added a project: Wikidata. TASK DETAILhttps://phabricator.wikimedia.org/T209621EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T190022: Separate the CirrusSearch/Elastic-specific code from Wikibase code base

2019-01-30 Thread Smalyshev
Smalyshev added a comment. We will also need to move options out. We could move it to set of WikibaseCirrusSearch options, or keep current option structure, in which case we'd need to enable extensions to define their own Wikibase options, which as I understand not the case now? Should we keep

[Wikidata-bugs] [Maniphest] [Updated] T213994: A Commons search user should be able to search for only captions

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added Discovery-Search; removed Discovery-Search (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T213994EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, dcausse, Jdforrester-WMF, Aklapper, Cparle

[Wikidata-bugs] [Maniphest] [Triaged] T214544: [Beta Commons]: All searches show a warning about mixing "entity and article namespaces"

2019-01-30 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority.Smalyshev edited projects, added Discovery-Search; removed Discovery-Search (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T214544EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: S

[Wikidata-bugs] [Maniphest] [Updated] T214544: [Beta Commons]: All searches show a warning about mixing "entity and article namespaces"

2019-01-30 Thread Smalyshev
Smalyshev added a comment. So searching article & entity namespace together is not supported (see T194968: Enable search in all wikidata namespaces combined for task about it). So, we have to either configure search so it by default searches only one kind of namespaces, or solve the

[Wikidata-bugs] [Maniphest] [Updated] T214544: [Beta Commons]: All searches show a warning about mixing "entity and article namespaces"

2019-01-30 Thread Smalyshev
Smalyshev added a subtask: T194968: Enable search in all wikidata namespaces combined. TASK DETAILhttps://phabricator.wikimedia.org/T214544EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Abit, Cparle, Ramsey-WMF, EBernhardson, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T194968: Enable search in all wikidata namespaces combined

2019-01-30 Thread Smalyshev
Smalyshev added a parent task: T214544: [Beta Commons]: All searches show a warning about mixing "entity and article namespaces". TASK DETAILhttps://phabricator.wikimedia.org/T194968EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Jdfor

[Wikidata-bugs] [Maniphest] [Updated] T206613: Search of wikidata string property values using haswbstatement is case sensitive

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added Discovery-Search; removed Discovery-Search (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T206613EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Multichill, Aklapper, Lydia_Pintscher, aude, debt

[Wikidata-bugs] [Maniphest] [Commented On] T214980: Support mathematical formulae in Wikidata Query Service UI on all browsers

2019-01-30 Thread Smalyshev
Smalyshev added a comment. I am not sure whether it's ok to use cdnjs.cloudflare.com for privacy reasons. AFAIK, so far we have avoided directly including non-WMF resources.TASK DETAILhttps://phabricator.wikimedia.org/T214980EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T214851: query.wikidata.org displays the TeX string rather than the MathML representation for some formulae

2019-01-30 Thread Smalyshev
Smalyshev added a comment. OK, looks fine for me in Firefox. We might want to create a new task for eventually adding _javascript_-based mathml rendering - like it or not, Chrome becomes more and more prominent (even Microsoft moved to use Chrome engine) and formulas look so much better

[Wikidata-bugs] [Maniphest] [Updated] T214851: query.wikidata.org displays the TeX string rather than the MathML representation for some formulae

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added Wikidata Query UI; removed Wikidata-Query-Service.Smalyshev added a comment. Hmm, I updated the entities, and I still see TeX there. However, looking at raw backend output, I do see mathml - check out the result of https://query.wikidata.org/sparql?query=DESCRIBE

[Wikidata-bugs] [Maniphest] [Updated] T214851: query.wikidata.org displays the TeX string rather than the MathML representation for some formulae

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added Wikidata-Query-Service; removed Wikidata Query UI.Smalyshev added a comment. OK, it looks like some entities indeed have Tex strings, even though export returns MathML now. Not sure when it changed - the entities have right timestamps, but the RDF is different

[Wikidata-bugs] [Maniphest] [Updated] T214851: query.wikidata.org displays the TeX string rather than the MathML representation for some formulae

2019-01-30 Thread Smalyshev
Smalyshev edited projects, added Wikidata Query UI; removed Wikidata-Query-Service. TASK DETAILhttps://phabricator.wikimedia.org/T214851EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Physikerwelt, SmalyshevCc: Lucas_Werkmeister_WMDE, Aklapper, Physikerwelt

[Wikidata-bugs] [Maniphest] [Closed] T213305: upgrade prometheus-blazegraph-exporter to python3

2019-01-29 Thread Smalyshev
Smalyshev closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T213305EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Mathew.onipe, SmalyshevCc: gerritbot, Mathew.onipe, Gehel, Aklapper, Legado_Shulgin, CucyNoiD, Nandana, Ne

[Wikidata-bugs] [Maniphest] [Updated] T207676: Increase deployment window of wdqs or parallelize scap deployment

2019-01-29 Thread Smalyshev
Smalyshev edited projects, added Discovery-Wikidata-Query-Service-Sprint; removed Discovery-Search (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T207676EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Gehel, Mathew.onipe

[Wikidata-bugs] [Maniphest] [Updated] T207837: wdqs updater should be better isolated from blazegraph and common workload should be shared between servers

2019-01-29 Thread Smalyshev
Smalyshev removed a project: Discovery-Search (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T207837EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Aklapper, Joe, Gehel, Legado_Shulgin, Nandana, thifranc, AndyTan

[Wikidata-bugs] [Maniphest] [Unassigned] T55652: Special:Search doesn't use labels and descriptions for suggestions but just the item ID

2019-01-29 Thread Smalyshev
Smalyshev removed Smalyshev as the assignee of this task. TASK DETAILhttps://phabricator.wikimedia.org/T55652EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Smalyshev, Wikidata-bugs, Yair_rand, Lydia_Pintscher, daniel, CucyNoiD

[Wikidata-bugs] [Maniphest] [Claimed] T200612: Wikidata's SPARQL endpoint doesn't escape commas in IRIs in CSV output

2019-01-29 Thread Smalyshev
Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T200612EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, seav, Aklapper, jindrichmynarz, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden

[Wikidata-bugs] [Maniphest] [Triaged] T199887: Client equivalent of haswbstatement

2019-01-29 Thread Smalyshev
Smalyshev triaged this task as "Low" priority. TASK DETAILhttps://phabricator.wikimedia.org/T199887EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: debt, Smalyshev, Aklapper, Bugreporter, Nandana, Lahi, Gq86, GoranSMilovanovic, QZand

[Wikidata-bugs] [Maniphest] [Updated] T194401: Investigate storing commons data in BlazeGraph

2019-01-29 Thread Smalyshev
Smalyshev edited projects, added Wikidata-Query-Service; removed Discovery-Search. TASK DETAILhttps://phabricator.wikimedia.org/T194401EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Cparle, SmalyshevCc: Tpt, Jheald, Aklapper, Smalyshev, Ramsey-WMF, Cparle

[Wikidata-bugs] [Maniphest] [Triaged] T209859: Wikidata autocomplete (wbsearchentities) results with score <= 0

2019-01-29 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAILhttps://phabricator.wikimedia.org/T209859EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: dcausse, Smalyshev, EBernhardson, Aklapper, Nandana, Lahi, Gq86, GoranSMilovanovi

[Wikidata-bugs] [Maniphest] [Commented On] T214680: Document statement URI format for RDF

2019-01-28 Thread Smalyshev
Smalyshev added a comment. Well, yes and no. For example, if we used sha1(statment id) as RDF statement URI, it would not technically be one-to-one but in reality we'd have no clashes, probably, and thus it would be OK for RDF. In the same vein, something like preg_replace( '/[^\w

[Wikidata-bugs] [Maniphest] [Commented On] T214260: Get value of page_image_free using MWAPI

2019-01-28 Thread Smalyshev
Smalyshev added a comment. OK, this is my fault, I misunderstood the original question. The situation is as follows: By default, the values are taken from the template You can override template values by specifying your own For ppprop, the properties that are specified (and only those

[Wikidata-bugs] [Maniphest] [Commented On] T190022: Separate the CirrusSearch/Elastic-specific code from Wikibase code base

2019-01-27 Thread Smalyshev
Smalyshev added a comment. OK, I am dealing with main Wikibase code so far, but would appreciate team's feedback as it would be time to move on to the extensions soon. Please tag whoever else on the team may be interested. So far I understand the following use Elastic functionality

[Wikidata-bugs] [Maniphest] [Commented On] T214680: Document statement URI format for RDF

2019-01-26 Thread Smalyshev
Smalyshev added a comment. So I would like to see the id conversion documented in BOTH directions This is a bit trickier. The reason is that RDF has particular requirements for URIs, which makes it much easier if the URI does not contain "bad" characters - among which is '$'. However,

[Wikidata-bugs] [Maniphest] [Commented On] T214260: Get value of page_image_free using MWAPI

2019-01-25 Thread Smalyshev
Smalyshev added a comment. I was under the impression that MWAPI sends the query basically unmodified No, it uses the template, which is documented at https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Individual_service_configuration and https://www.mediawiki.org/wiki

[Wikidata-bugs] [Maniphest] [Edited] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-24 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...Immediate: *[x] Copy database from wdq[345] to wdq7 and wdq8 *[x] Restore updates on wdq7 and wdq8 *[x] Collect allocator stats everywhere and see which servers are also in danger *[x] Write an incident report

[Wikidata-bugs] [Maniphest] [Edited] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-24 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTIONAs described in {T213134}, we've approached - and on some servers, exceeded - the allocator limit on Blazegraph. This task will aggregate all tasks we need to perform to solve the issue. Related incident report

[Wikidata-bugs] [Maniphest] [Unblock] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-24 Thread Smalyshev
Smalyshev closed subtask T213372: Create monitoring of allocators as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T213210EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, Aklapper, Smalyshev, Nandana,

[Wikidata-bugs] [Maniphest] [Closed] T213372: Create monitoring of allocators

2019-01-24 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T213372EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Mathew.onipe, Aklapper, Gehel, Smalyshev, CucyNoi

[Wikidata-bugs] [Maniphest] [Closed] T206108: Limit CPU consumption of blazegraph

2019-01-24 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T206108EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Aklapper, Gehel, Nandana, Lahi, Gq86, Lucas_Werkme

[Wikidata-bugs] [Maniphest] [Commented On] T214260: Get value of page_image_free using MWAPI

2019-01-24 Thread Smalyshev
Smalyshev added a comment. But then why does the API URL provided in the task description (link) return a wikibase_item with or without =wikibase_item|page_image_free? Probably by default without ppprop the API returns all of them. But MWAPI sets ppprop=wikibase_item. It’s only missing

[Wikidata-bugs] [Maniphest] [Commented On] T214260: Get value of page_image_free using MWAPI

2019-01-23 Thread Smalyshev
Smalyshev added a comment. Add this to query: bd:serviceParam mwapi:ppprop "wikibase_item|page_image_free" . By default, only wikibase_item is fetched (see https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/mwapi.json).TASK DETAILhttps://phabricator.wikimedia.org/T2

[Wikidata-bugs] [Maniphest] [Closed] T214260: Get value of page_image_free using MWAPI

2019-01-23 Thread Smalyshev
Smalyshev closed this task as "Invalid". TASK DETAILhttps://phabricator.wikimedia.org/T214260EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Smalyshev, Aklapper, Wikidata-bugs, Luitzen, Nandana, Lahi, Gq86, Lucas_Werkme

[Wikidata-bugs] [Maniphest] [Updated] T115788: Wikidata query endpoint shouldn't return traceback when an error is encountered

2019-01-23 Thread Smalyshev
Smalyshev removed a project: Security.Smalyshev added a comment. I think we can drop #security since I don't see any real security impact here. Please re-tag if you feel there's any security issue still present.TASK DETAILhttps://phabricator.wikimedia.org/T115788EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Updated] T213628: change link text in embedded query flyout

2019-01-23 Thread Smalyshev
Smalyshev edited projects, added Wikidata Query UI; removed Wikidata-Query-Service. TASK DETAILhttps://phabricator.wikimedia.org/T213628EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Jsc39, Greta_Doci_WMDE, Charlie_WMDE, Jan_Dittrich

[Wikidata-bugs] [Maniphest] [Closed] T213213: Duplicate normalized triples for values

2019-01-23 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task.Smalyshev added a comment. Closing as resolved for now, since there's not much to do, most likely solving T167759 is needed to prevent re-occurrences.TASK DETAILhttps://phabricator.wikimedia.org/T213213EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Commented On] T197583: Decommission wikidata-lexeme demo system

2019-01-23 Thread Smalyshev
Smalyshev added a comment. Perhaps we could create a script in Lexeme to create a base set of items and lexemes for use in testing? That is a great idea, would also allow people like me to set up my own test environment without bothering anybody else.TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T115788: Wikidata query endpoint shouldn't return traceback when an error is encountered

2019-01-23 Thread Smalyshev
Smalyshev added a comment. The issue is, as you noted, that backtraces are sometimes useful. And it's easy to ignore the part that you don't need, but it's hard to restore the part you do need if it's not there.TASK DETAILhttps://phabricator.wikimedia.org/T115788EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T115788: Wikidata query endpoint shouldn't return traceback when an error is encountered

2019-01-23 Thread Smalyshev
Smalyshev added a comment. Third-parties who run this code may end up unintentionally exposing backend information in this uncaught exception It's only exceptions (which are meant to be public) and class names/line numbers, which are part of open source code, so are public too. So I don't see any

[Wikidata-bugs] [Maniphest] [Claimed] T190022: Separate the CirrusSearch/Elastic-specific code from Wikibase code base

2019-01-22 Thread Smalyshev
Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T190022EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Addshore, Aleksey_WMDE, thiemowmde, RazShuty, WMDE-leszek, Lydia_Pintscher, Aklapper, Smalyshev, ET4Eva

[Wikidata-bugs] [Maniphest] [Updated] T213372: Create monitoring of allocators

2019-01-22 Thread Smalyshev
Smalyshev added a project: Discovery-Wikidata-Query-Service-Sprint. TASK DETAILhttps://phabricator.wikimedia.org/T213372EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Mathew.onipe, Aklapper, Gehel, Smalyshev, CucyNoiD, Nandana

[Wikidata-bugs] [Maniphest] [Commented On] T213913: Retire or update wikidata-dev/wikibase-lexeme trusty cloud VPS instance

2019-01-21 Thread Smalyshev
Smalyshev added a comment. I wish I was pinged beforehand for it, but I guess that's too late now.TASK DETAILhttps://phabricator.wikimedia.org/T213913EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: WMDE-leszek, SmalyshevCc: Lucas_Werkmeister_WMDE, Smalyshev

[Wikidata-bugs] [Maniphest] [Closed] T204267: Flood of WDQS requests from wbqc

2019-01-18 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T204267EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Tpt, Pintoch, Stashbot, Jonas, Addshore,

[Wikidata-bugs] [Maniphest] [Commented On] T210495: Number of Senses is decreasing on ListeriaBot's report

2019-01-17 Thread Smalyshev
Smalyshev added a comment. OK the unicode thing is fixed now, so I wonder if there are any other issues there.TASK DETAILhttps://phabricator.wikimedia.org/T210495EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Liuxinyu970226, Smalyshev

[Wikidata-bugs] [Maniphest] [Updated] T213994: A Commons search user should be able to search for captions

2019-01-17 Thread Smalyshev
Smalyshev added a comment. Currently on wikibase we can query the entity labels or we can perform a normal query, but we can't integrate the two result sets into a single list. Yes, this is essentially what T194968: Enable search in all wikidata namespaces combined and T190454: Display entity

[Wikidata-bugs] [Maniphest] [Commented On] T214032: query service should send a retry-after header with a 429 response

2019-01-17 Thread Smalyshev
Smalyshev added a comment. I thought we do send Retry-After. Can we get a reproducing case? Maybe some proxy drops it? Because I am looking at the throttling code and clearly see producing the Retry-After header.TASK DETAILhttps://phabricator.wikimedia.org/T214032EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Closed] T213212: Move category namespace to a separate blazegraph instance

2019-01-16 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, gerritbot, Aklapper, Smalyshev, CucyNoi

[Wikidata-bugs] [Maniphest] [Unblock] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-16 Thread Smalyshev
Smalyshev closed subtask T213212: Move category namespace to a separate blazegraph instance as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T213210EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, Aklapper,

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-16 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Update categories alias to new namespace *[x] Drop categories namespace on main instanceTASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T213212: Move category namespace to a separate blazegraph instance

2019-01-16 Thread Smalyshev
Smalyshev added a comment. Daily reload seems to have gone fine, next step is deleting existing namespaces.TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, gerritbot, Aklapper

[Wikidata-bugs] [Maniphest] [Commented On] T213889: WDQS UI fails on large queries

2019-01-16 Thread Smalyshev
Smalyshev added a comment. I see the code is: if ( request.getAllResponseHeaders() === '' ) { // query might have been too long for GET, retry with POST settings.method = 'POST'; $.ajax( self._serviceUri, settings ).done( done ).fail( fail ); } I suspect when the server rejects

[Wikidata-bugs] [Maniphest] [Commented On] T213889: WDQS UI fails on large queries

2019-01-16 Thread Smalyshev
Smalyshev added a comment. Well, maybe there's some gap between client-side and server-side limits?TASK DETAILhttps://phabricator.wikimedia.org/T213889EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Lucas_Werkmeister_WMDE, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-16 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Update cron scripts in puppet to use new endpoint *[x] Update configs with CATEGORY_ENDPOINT pointing to new instance. Note that this will also switch crons to update new instance, so better to do far away

[Wikidata-bugs] [Maniphest] [Unblock] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-16 Thread Smalyshev
Smalyshev closed subtask T213854: Reload database on wdq2[456] from another server as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T213210EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, Aklapper, Smalyshe

[Wikidata-bugs] [Maniphest] [Closed] T213854: Reload database on wdq2[456] from another server

2019-01-16 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T213854EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Aklapper, Mathew.onipe, Gehel, Smalyshev, Nandana,

[Wikidata-bugs] [Maniphest] [Created] T213889: WDQS UI fails on large queries

2019-01-15 Thread Smalyshev
Smalyshev created this task.Smalyshev added a project: Wikidata Query UI.Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata. TASK DESCRIPTIONIf you put a large query into WDQS GUI, it would produce 414 Request-URI Too Large since URI size

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-15 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...* Update categorload delta dailies alias to new namespacethat accumulated since initial load * Upload delta dailies that accumulated since initial load* Switch nginx to use new instance (nginx config patch

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-15 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Data_reload_procedure | Load weeklies and dailies ]] [x] Update cron scripts in puppet to use new endpoint * Update configs with CATEGORY_ENDPOINT

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-15 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...[x] Create new namespace on category instance [x] [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Data_reload_procedure | Load weeklies and dailies ]] * Update configs with CATEGORY_ENDPOINT pointing

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-15 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...* Update categories alias to new namespace * Upload delta dailies that accumulated since initial load * Switch nginx to use new instance (nginx config patch)...TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Triaged] T213854: Reload database on wdq2[456] from another server

2019-01-15 Thread Smalyshev
Smalyshev triaged this task as "High" priority. TASK DETAILhttps://phabricator.wikimedia.org/T213854EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Aklapper, Mathew.onipe, Gehel, Smalyshev, Nandana, Lahi, Gq86, Lucas_Werkme

[Wikidata-bugs] [Maniphest] [Updated] T213210: WDQS is hitting allocator limit on Blazegraph

2019-01-15 Thread Smalyshev
Smalyshev added a subtask: T213854: Reload database on wdq2[456] from another server. TASK DETAILhttps://phabricator.wikimedia.org/T213210EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Gehel, Aklapper, Smalyshev, Nandana, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T213854: Reload database on wdq2[456] from another server

2019-01-15 Thread Smalyshev
Smalyshev added a parent task: T213210: WDQS is hitting allocator limit on Blazegraph. TASK DETAILhttps://phabricator.wikimedia.org/T213854EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Aklapper, Mathew.onipe, Gehel, Smalyshev, Nandana, Lahi

[Wikidata-bugs] [Maniphest] [Created] T213854: Reload database on wdq2[456] from another server

2019-01-15 Thread Smalyshev
Smalyshev created this task.Smalyshev added a project: Wikidata-Query-Service.Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata. TASK DESCRIPTIONWe have low allocator count on these hosts too: wdq24 261930 214 wdq25 262090 54 wdq26 261989 155

[Wikidata-bugs] [Maniphest] [Changed Project Column] T213213: Duplicate normalized triples for values

2019-01-15 Thread Smalyshev
Smalyshev moved this task from Doing to Done on the User-Smalyshev board.Smalyshev added a comment. I've cleaned up the bad values, but still unclear how they came into being and how to prevent this from happening again in the future.TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T213212: Move category namespace to a separate blazegraph instance

2019-01-14 Thread Smalyshev
Smalyshev added a comment. Category data loaded up to 20190114 on all servers. We'll probably need to update to 20190115 manually since crons are still on old namespace.TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-14 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...* [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Data_reload_procedure | Load weeklies and dailies ]] * Update categories alias to new namespace * Switch nginx to use new instance (nginx config

[Wikidata-bugs] [Maniphest] [Edited] T213212: Move category namespace to a separate blazegraph instance

2019-01-14 Thread Smalyshev
Smalyshev updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...Migration plan after we have 2 instances running: *[x] Create new namespace on category instance...TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] [Closed] T213234: Create puppet config to run two instances of Blazegraph

2019-01-14 Thread Smalyshev
Smalyshev closed this task as "Resolved".Smalyshev claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T213234EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, gerritbot, Mathew.onipe, Gehel, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Unblock] T213212: Move category namespace to a separate blazegraph instance

2019-01-14 Thread Smalyshev
Smalyshev closed subtask T213234: Create puppet config to run two instances of Blazegraph as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Aklapper, Smalyshe

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

2019-01-14 Thread Smalyshev
Smalyshev added a comment. I've tried to read all of it and maybe I've missed something, but I am still not sure what added value having such separate service gives us. We are creating pretty complex patten of interaction between Mediawiki and outside service, and I am not sure why this service

[Wikidata-bugs] [Maniphest] [Updated] T213212: Move category namespace to a separate blazegraph instance

2019-01-14 Thread Smalyshev
Smalyshev edited projects, added User-Smalyshev; removed Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T213212EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Aklapper, Smalyshev, Nandana, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T213234: Create puppet config to run two instances of Blazegraph

2019-01-14 Thread Smalyshev
Smalyshev added a project: User-Smalyshev. TASK DETAILhttps://phabricator.wikimedia.org/T213234EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: gerritbot, Mathew.onipe, Gehel, Aklapper, Smalyshev, CucyNoiD, Nandana, NebulousIris, Gaboe420

[Wikidata-bugs] [Maniphest] [Updated] T213213: Duplicate normalized triples for values

2019-01-14 Thread Smalyshev
Smalyshev added a project: User-Smalyshev.Smalyshev added a comment. I've cleaned up some of the bad values (looks like may be a fallout from float precision issue) but there are still a bunch of values with duplicate normalized values (as above) - 17398 to be exact.TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Commented On] T212822: "Unknown prefix: mwapi" JS errors in console

2019-01-14 Thread Smalyshev
Smalyshev added a comment. If you use an undefined prefix anywhere in your query, that’s an error, even if it’s only inside a SERVICE Hmm, I'm not sure this is right... But maybe it's a limitation since SPARQL parser can't know whether it's internal or external service. I need to check

<    4   5   6   7   8   9   10   11   12   13   >