[Wikidata-bugs] [Maniphest] T341957: Wrong number of Wikibase entities in NewPP limit report
RolandUnger created this task. RolandUnger added projects: MediaWiki-Parser, Wikidata, Wikidata Analytics. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION **Steps to replicate the issue** (include links if applicable): - Use for instance "https://de.wikivoyage.org/wiki/Wien"; or "https://de.wikivoyage.org/wiki/Halle_(Saale)" - Open the page source code (html). Move to "NewPP limit report" - Look for "Number of Wikibase entities loaded" **What happens?**: - Mostly, only 1/400 or 2/400 are shown which is too little **What should have happened instead?**: - In case of "https://de.wikivoyage.org/wiki/Halle_(Saale)" about 160/400 should be shown as it was before. **Software version** (skip for WMF-hosted wikis like Wikipedia): **Other information** (browser name/version, screenshots, etc.): - Firefox 115.0.2 (64-Bit), Win 10 TASK DETAIL https://phabricator.wikimedia.org/T341957 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Aklapper, RolandUnger, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, JJMC89, _jensen, rosalieper, Scott_WUaS, Nirmos, Cwek, Wikidata-bugs, aude, Dinoguy1000, Arlolra, Jackmcbarn, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T309971: mw.wikibase.getBestStatements() should not return values which are definitely wrong
RolandUnger created this task. RolandUnger added projects: Wikibase and Wikidata Architecture Overview, Wikidata, Wikibase-Lua. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION **List of steps to reproduce** (step by step, including full links if applicable): - See for instance - https://www.wikidata.org/w/index.php?title=Q126514&diff=prev&oldid=1655511851 - https://www.wikidata.org/w/index.php?title=Q637182&curid=599468&diff=1655528424&oldid=1655084002 - Maybe the values in question were imported by faulty bot scripts. **What happens?:** - If these values are used in a Lua script, they can cause runtime errors because the programmers expect to get only checked values. **What should have happened instead?**: - In any case, mw.wikibase.getBestStatements() should return only checked values because they should be the best ones. **Possible ways to treat values which are definitely wrong:** - Refuse the upload of wrong values, - Downgrade theses values from normal or preferred rank to low rank, - Check against regex rules, - Use additional functions which are more effective than regex rules, for instance for url and email addresses. - At de-voy, in case of urls we are using for some modules the module Module:UrlCheck but this should not the preferred method to handle such wrong values because it is time-consuming. **Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc.**: - Wikivoyage, Firefox 100 By the way, data quality plays an important role for general acceptance of Wikidata, especially in the Wikipedias. TASK DETAIL https://phabricator.wikimedia.org/T309971 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Aklapper, RolandUnger, LennardHofmann, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T265562: [6hr.] Investigation - Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. I was surprised, too. And I did not made any changes (yesterday morning I only added some maintenance categories). The computing times for `addStatementUsage` oscillate (between 400 and 900 ms) but they are clearly reduced. I think yesterday the 1.36.0-wmf.14 Mediawiki branch was deployed at Wikivoyage. Maybe that means to find the cause we had to look for it outside of Wikibase. Unless I'm very much mistaken `addStatementUsage` only makes a registration of entities and properties used. This should not be expensive. But I have no idea what happens at the registration and what went wrong. As you can see: our main problem is Scribunto_LuaSandboxCallback::callParserFunction which takes between 40 and 50 % of total Lua time (250 expensive parser function calls). But this is a Kartographer problem and therefore another stuff. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: toan, RolandUnger Cc: Tarrow, Pablo-WMDE, Lucas_Werkmeister_WMDE, toan, hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: [6hr.] Investigation - Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. The computing times for `Scribunto_LuaSandboxCallback::getEntity` and `Scribunto_LuaSandboxCallback::addStatementUsage` did not change much, but only that of `Scribunto_LuaSandboxCallback::addStatementUsage` changed drastically. The number of properties fetched increased within the last four weeks only by one which cannot explain the increase of computing time of this simple function. Maybe there is a problem with the accumulator/cache or database. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: toan, RolandUnger Cc: Pablo-WMDE, Lucas_Werkmeister_WMDE, toan, hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: [6hr.] Investigation - Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. I am not sure if T263999 <https://phabricator.wikimedia.org/T263999> is the real cause for this bug. But I cannot exclude this cause, or maybe both bugs have a joint cause elsewhere. `addStatementUsage` only registers the use of an entity and a property and has maybe nothing to do with functions in which a language was specified. Today I added some additional code/maintenance to our vCard script to check the usage of getLabel and getLabelByLang Wikibase functions. Both functions are //not used// in the article of Halle (Saale) <https://de.wikivoyage.org/wiki/Halle_(Saale)> but in the article of Eisenbahnmuseen in Europa <https://de.wikivoyage.org/wiki/Eisenbahnmuseen_in_Europa>. In any case, the bug still occurs. Otherwise properties with type `monolingualtext` are used. There are several maintenance categories to check usage of getLabel: getLabel used <https://de.wikivoyage.org/wiki/Kategorie:VCard:_getLabel_benutzt> (temporarily added), label from Wikidata <https://de.wikivoyage.org/wiki/Kategorie:VCard:_Label_aus_Wikidata>, unknown label <https://de.wikivoyage.org/wiki/Kategorie:VCard:_Unbekanntes_Label_oder_Id>, and opening-hours label from Wikidata <https://de.wikivoyage.org/wiki/Kategorie:VCard:_%C3%96ffnungszeit-Label_aus_Wikidata>. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: toan, RolandUnger Cc: Pablo-WMDE, Lucas_Werkmeister_WMDE, toan, hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. The Wikivoyage template vCard <https://de.wikivoyage.org/wiki/Modul:VCard> can use up to 50 statements per template (see table ParWD at Module:VCard/Params <https://de.wikivoyage.org/wiki/Modul:VCard/Params>, too). About 5 `addStatementUsage` calls per template on average are not really much even for an entity which is already loaded (loading takes only about 2.5 ms). Within the last weeks we added only //one// statement (P3025 <https://phabricator.wikimedia.org/P3025>: open days). Therefore, the number of `addStatementUsage` calls increased only by a few percent. Some days ago a single `addStatementUsage` call costs // only about 1 ms // but now // 6 ms //. The question arose: why this drastic change: more than 500 %? I have no idea why an access to an entity table already loaded is // now // as much expensive. If we cannot solve this problem then Wikidata will become useless for Wikimedia projects especially for list articles. An optimized Wikidata access for Wikivoyage and other Wikimedia projects is a mission-critical goal. We need the data at Wikidata to easily maintain them for all Wikivoyage projects. The main problem at Wikivoyage is to use data not only in a single information box but at many places in text, too. And why we cannot use this data any longer as we did it within the last four years? When I started to work on a Wikidata usage in vCard template four years ago I was really astonished by the huge computing time: about one minute per article (of course, the parsing was stopped before). That's why I am using a lot of helper tables to // prevent // additional Wikidata access for instance to get labels and country-specific data. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Lucas_Werkmeister_WMDE, toan, hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. If the bug is caused by cache limitations: Maybe a simple destructor `entity:remove()` would help. People told me that all entities are cached until the end of article parsing. But most of the entities are used in only one template and must not be cached any longer. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. I tried to remove the museum section from https://de.wikivoyage.org/wiki/Halle_(Saale) and to stop using qualifiers and references from entities but the bug remained. But it seems that this bug depends on the number of entities loaded: - https://de.wikivoyage.org/wiki/Wernigerode: 33 entities: addStatementUsage not mentioned in Lua profile - https://de.wikivoyage.org/wiki/El_Gouna: 28 entities: addStatementUsage 180 ms (12 % of Lua computing time) - https://de.wikivoyage.org/wiki/St._Gallen: 62 entities: addStatementUsage 400 ms (14 %) - https://de.wikivoyage.org/wiki/Bremen: 104 entities: addStatementUsage 640 ms (16 %) - https://de.wikivoyage.org/wiki/Benutzer:Balou46/Testseite6: 124 entities: addStatementUsage 3020 ms (48 %) - https://de.wikivoyage.org/wiki/Benutzer:Balou46/Testseite5: 127 entities: addStatementUsage 2780 ms (47 %) - https://de.wikivoyage.org/wiki/Eisenbahnmuseen_in_Europa: 144 entities: addStatementUsage 6900 ms (69 %) - https://de.wikivoyage.org/wiki/Halle_(Saale): 166 entities: addStatementUsage 4220 ms (51 %) A week ago, the addStatementUsage computing time for the last case was about 600 ms (less than 10 %). All statements are retrieved with `entity:getBestStatements( p )` function in `Module:Wikidata_utilities`. The article entity is untouched. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. See also T189409 <https://phabricator.wikimedia.org/T189409>. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Anomie, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added projects: MediaWiki-extensions-WikibaseClient, MediaWiki-extensions-Scribunto. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, alex-mashin, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Anomie, Jackmcbarn, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. It seems that it is a sudden spike. The computing time only for `addStatementUsage` jumped from about 600 ms to 5000 ms (all Lua stuff from about 4.5 sec to about 9 sec). Within the last two weeks I added and optimized the analysis of opening hours to our vCard template. As in the past I used the same technique to //prevent// retrieving data from Wikidata. We are using several tables like this: https://de.wikivoyage.org/wiki/Modul:Hours/i18n with prepared data. On this way there is usually only one getEntity call per template. The number of entities used increased within the last two weeks only by about 1. Of course I added some statements to the Wikidata datasets (mainly of museums) to test the display of opening hours. Within the last two weeks there was no significant change. The core functions were added already on Sep. 26, 2020. There is another article with the same behavior: https://de.wikivoyage.org/wiki/Eisenbahnmuseen_in_Europa. For this article, I got today in the morning a runtime error that there was not enough time. Lua Profile: Scribunto_LuaSandboxCallback::addStatementUsage 5580 ms 68.7% Scribunto_LuaSandboxCallback::addSiteLinksUsage 800 ms 9.9% Scribunto_LuaSandboxCallback::callParserFunction 720 ms 8.9% Scribunto_LuaSandboxCallback::getEntity 320 ms 3.9% Scribunto_LuaSandboxCallback::addLabelUsage 120 ms 1.5% Scribunto_LuaSandboxCallback::incrementStatsKey 120 ms 1.5% Scribunto_LuaSandboxCallback::fullUrl 80 ms 1.0% Scribunto_LuaSandboxCallback::gsub40 ms 0.5% init 40 ms 0.5% ? 40 ms 0.5% [others] I think there is a cache problem (see also T236749 <https://phabricator.wikimedia.org/T236749>). And I assume that we exceeded a threshold with only minor additions. As already stated I think this bug is the same like T236485 <https://phabricator.wikimedia.org/T236485>. The caching bug was not really solved but only downplayed. Therefore it was only a question of time that this bug reoccured. There are only 166 entity calls (one per template) in the article of Halle (Saale) which is clearly less than the limit of 400. And I hope that we can use up to 250 Wikidata calls in future. Of course we use a lot of properties (in article mentioned about 50, maximally about 100) but we do not retrieve additional ones because of the tables mentioned. (I remember that you stated that Wikivoyage is the Wikimedia site which uses Wikidata in most extreme extent -- more than Commons. That is really true.) TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, Wikidata-bugs, aude, GWicke, Dinoguy1000, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a project: MediaWiki-Cache. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: hoo, Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, Wikidata-bugs, aude, GWicke, Dinoguy1000, Mbch331, Rxy, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265565: Getting Q-Id info with Lua call
RolandUnger created this task.
RolandUnger added projects: Wikidata, Wikibase-Lua.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
It is impossible to access Q-Id info with a Lua function. It can be done only
by a API call like
`https://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q42&props=info`
It returns:
{
"entities": {
"Q42": {
"pageid": 138,
"ns": 0,
"title": "Q42",
"lastrevid": 503604471,
"modified": "2017-06-20T10:01:26Z",
"type": "item",
"id": "Q42"
}
},
"success": 1
}
The most important is of course the `modified` parameter. I think a function
should return the complete info block.
TASK DETAIL
https://phabricator.wikimedia.org/T265565
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RolandUnger
Cc: RolandUnger, Aklapper, Akuckartz, darthmon_wmde, Nandana, lucamauri, Lahi,
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS,
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger added a comment. Maybe it is the same like T236485 <https://phabricator.wikimedia.org/T236485>. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, Wikidata-bugs, aude, GWicke, Dinoguy1000, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265562: Wikibase Lua: addStatementUsage seems unjustifiably expensive
RolandUnger created this task. RolandUnger added projects: Wikidata, Wikibase-Lua, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Performance Issue. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION The computing times increased drastically by about 100 %, mainly produced by a drastic increase of the computing time of the Lua function addStatementUsage (increase of about 500 %). From the Lua profile of https://de.wikivoyage.org/wiki/Halle_(Saale) (see html source code): Lua Profile: Scribunto_LuaSandboxCallback::addStatementUsage 4540 ms 55.0% Scribunto_LuaSandboxCallback::callParserFunction1640 ms 19.9% Scribunto_LuaSandboxCallback::getEntity 540 ms 6.5% Scribunto_LuaSandboxCallback::addSiteLinksUsage 420 ms 5.1% Scribunto_LuaSandboxCallback::getEntityStatements320 ms 3.9% init100 ms 1.2% recursiveClone100 ms 1.2% ?100 ms 1.2% Scribunto_LuaSandboxCallback::gsub80 ms 1.0% init80 ms 1.0% [others] 340 ms 4.1% Maybe it is the same like T236485. TASK DETAIL https://phabricator.wikimedia.org/T265562 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Lydia_Pintscher, RolandUnger, Aklapper, Akuckartz, Iflorez, darthmon_wmde, alaa_wmde, Nandana, lucamauri, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Jonas, Volker_E, Wikidata-bugs, aude, GWicke, Dinoguy1000, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T237884: mw.wikibase.getBestStatements is slow when used on objects with many properties
RolandUnger added a comment. In this regard I made the proposal of T179638 <https://phabricator.wikimedia.org/T179638> about two years ago. TASK DETAIL https://phabricator.wikimedia.org/T237884 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: RolandUnger, Addshore, Lucas_Werkmeister_WMDE, Aklapper, Reinhard_Mueller, Liuxinyu970226, darthmon_wmde, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Volker_E, Wikidata-bugs, aude, GWicke, Dinoguy1000, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T243779: Wikidata internal error
RolandUnger added a project: Wikibase-DataModel. TASK DETAIL https://phabricator.wikimedia.org/T243779 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: RolandUnger, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T243779: Wikidata internal error
RolandUnger created this task. RolandUnger added a project: Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION If I call `https://www.wikidata.org/w/index.php?title=Q28966042&redirect=no` then I get a server error or an exception like `[Xi8HgwpAADsAADteSMgAAACA] 2020-01-27 15:53:39: Fataler Ausnahmefehler des Typs „Wikibase\DataModel\Services\Lookup\UnresolvedEntityRedirectException“`. Wikimedia error message: "Error Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes. See the error message at the bottom of this page for more information." TASK DETAIL https://phabricator.wikimedia.org/T243779 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: RolandUnger, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T225497: Update Wikibase Lua documentation
RolandUnger added a comment. After ten weeks nothing is done. Why? TASK DETAIL https://phabricator.wikimedia.org/T225497 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: WMDE-leszek, Aklapper, Lydia_Pintscher, RolandUnger, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, Jayprakash12345, QZanden, LawExplorer, _jensen, rosalieper, Jonas, Wikidata-bugs, aude, Gryllida, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T66315: Move "Data item" link outside of sidebar toolbox
RolandUnger added a comment. @WMDE-leszek, for instance on the French Wikipedia and on the German Wikivoyage we added icons to the items of the "in other project" list to make them more prominent. That's why we added rules to the MediaWiki:Vector.css stylesheet. And that's why I asked to inform the communities to think about additions to other stylesheets. @Lea_Lacroix_WMDE, because of the rollback I cannot give an example any longer. But we observed these failures both yesterday and today. I know that a purging is possible instead a zero-edit. But both actions have to be done manually by administrators or other authors of many projects which means a huge work for them. You should keep smaller communities in mind because they have not the manpower to read all messages in the Wikimedia universe, to administrate and keep the Wikis running. And who are really angry that the Wikidata team is not doing their job like in case of T225497 <https://phabricator.wikimedia.org/T225497>. TASK DETAIL https://phabricator.wikimedia.org/T66315 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, RolandUnger Cc: RolandUnger, WMDE-leszek, Janbery, Znotch190711, Lejonel, Ladsgroup, alaa_wmde, Lea_Lacroix_WMDE, noarave, Soluvo, Addshore, TerraCodes, Izno, Aklapper, Nemo_bis, Ricordisamoa, MZMcBride, Quiddity, Lydia_Pintscher, Hook696, Daryl-TTMG, RomaAmorRoma, 0010318400, E.S.A-Sheild, darthmon_wmde, joker88john, Viztor, DannyS712, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, JJMC89, Maathavan, _jensen, rosalieper, Jonas, Johan, Luke081515, Wikidata-bugs, aude, TheDJ, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T230958: deWikisource doesn't show Wikibase-Item (API) / Link to Wikidata (UI)
RolandUnger added a comment. This behavior occurs in other projects, too. I mentioned this in T66315 <https://phabricator.wikimedia.org/T66315>. Unfortunately it is now necessary to make zero-edits to show the link in the "in other projects" list. TASK DETAIL https://phabricator.wikimedia.org/T230958 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: RolandUnger, Aklapper, LibrErli, darthmon_wmde, MattLongCT, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, Mahir256, QZanden, LawExplorer, _jensen, rosalieper, Bodhisattwa, Samwilson, Wikidata-bugs, aude, jayvdb, Shizhao, Billinghurst, Mbch331, jayantanth, Krenair ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T66315: Move "Data item" link outside of sidebar toolbox
RolandUnger added a comment. You should inform the communities on this (skin) change. This is necessary because the toolbox item is already vanished but the item in the "other project" list not yet available. We had to make zero-edits to make the new item visible. It should be mentioned that the "`t-wikibase`" id is not only used in style sheets but in JavaScript scripts, too. Maybe the new `wb-otherproject-wikibase-item` class has to be added to other style sheets. The items in the other projects list are not ordered alphabetically, at least in non-English branches. TASK DETAIL https://phabricator.wikimedia.org/T66315 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, RolandUnger Cc: RolandUnger, WMDE-leszek, Janbery, Znotch190711, Lejonel, Ladsgroup, stjn, alaa_wmde, Lea_Lacroix_WMDE, noarave, Soluvo, Addshore, TerraCodes, Izno, Aklapper, Nemo_bis, Ricordisamoa, MZMcBride, Quiddity, Lydia_Pintscher, Hook696, Daryl-TTMG, RomaAmorRoma, 0010318400, E.S.A-Sheild, darthmon_wmde, joker88john, Viztor, DannyS712, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, JJMC89, Maathavan, _jensen, rosalieper, Jonas, Johan, Luke081515, Wikidata-bugs, aude, TheDJ, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T226925: mw.wikibase.getReferencedEntityId() fails
RolandUnger added a comment. Missing Lua documentation is a big problem, see T225497 <https://phabricator.wikimedia.org/T225497>, too. TASK DETAIL https://phabricator.wikimedia.org/T226925 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RolandUnger Cc: Lydia_Pintscher, RolandUnger, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, Jayprakash12345, QZanden, LawExplorer, _jensen, rosalieper, Jonas, Wikidata-bugs, aude, Gryllida, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T226925: mw.wikibase.getReferencedEntityId() fails
RolandUnger created this task.
RolandUnger added projects: Wikidata, Wikidata-Campsite, Wikibase-Lua,
MediaWiki-Documentation.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
Sometimes mw.wikibase.getReferencedEntityId() calls unexpectedly fail. For
instance, a single
`mw.wikibase.getReferencedEntityId( 'Q637739', 'P31', { 'Q7397', 'Q2095',
'Q55488' } )`
call gives the German result
`Lua-Fehler: Zu viele Aufrufe von „mw.wikibase.getReferencedEntityId“, nur
bis zu 3 sind erlaubt.`
(Too many mw.wikibase.getReferencedEntityId calls. Only three are allowed.)
I do not know what happened because it is only a single call, and there are
only three P31 <https://phabricator.wikimedia.org/P31> values. What can be done
to prevent or to catch this error message?
This function restriction is not documented in the Wikibase documentation at
https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua.
TASK DETAIL
https://phabricator.wikimedia.org/T226925
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RolandUnger
Cc: RolandUnger, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86,
GoranSMilovanovic, Jayprakash12345, QZanden, LawExplorer, _jensen, rosalieper,
Jonas, Wikidata-bugs, aude, Gryllida, Lydia_Pintscher, Mbch331, Jay8g
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T182147: more convenience functions for Lua
RolandUnger added a comment. At Wikivoyage (Modul:FastWikidata) we introduced some helpful and simple functions with support of @thiemowmde. Searching for parent entities along a P31-P279 chain. Proposal: typeSearch( childId, idArray, limit ). childId is the entity id (or entity) to start the search. idArray contains P31 or P279 ids to look for. limit gives the maximum count of parent levels to search. typeSearch gives nil or a q id from the array. We need this search to get more general instances or classes. For instance: Q320366: we want to know that it is a train station (Q55488) instead of an interchange station (Q1147171). A similar question is to ask for a state (see above) or first administrative division starting from any location. We need a function providing an array with the first, n or all properties together with its qualifier values or ids. The qualifier itself is known. getPropertyWithQualifyer( id, p, q ). The result could be an array with items consisting of the value and a qualifier values array. TASK DETAILhttps://phabricator.wikimedia.org/T182147EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: RolandUnger, Agabi10, RexxS, Vriullop, MisterSynergy, Ghuron, Uzume, putnik, Jonas, aude, hoo, Ladsgroup, Tpt, thiemowmde, eranroz, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Wikidata-bugs, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. Thanks again to @Anomie for the profiler details. The fixes made by @thiemowmde could help to reduce the computing time of the Wikibase related calls from former 5000 ms to 900 ms now. I agree with thiemowmde to investigate further improvements in frame of T182147. I propose to separate the problem of expensive parser-function calls to another task. Most of these calls are made to create the Kartographer extension tags. That's why I created the task T189769. From my point of view this seems to be the best way to prevent expensive parser-function calls. We can close this task if you agree with the separation of the Kartographer calls.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, Jrbranaa, QZanden, LawExplorer, Vali.matei, SundanceRaphael, Volker_E, Wong128hk, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. Today the computing time was similar to the values a few weeks ago. For https://de.wikivoyage.org/wiki/Halle_%28Saale%29, we have now 3700-3900 ms as compared to about yesterday's 8000 ms for 237 vCard templates. Comparing with other articles I estimated a computing time of about 11 ms for a vCard without Wikidata access and about 20 ms for a vCard with Wikidata access. The difference seems to be mainly attributed to mw.wikibase.getEntity( id ) calls.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jrbranaa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Maathavan, SundanceRaphael, Volker_E, Wong128hk, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. @thiemowmde Of course I can wait until Wednesday/Thursday to test it again. The report will follow.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jrbranaa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. Thanks very much for your answers. As I told now parsing takes now between 8 to 10 s, a few weeks ago it was only 4 s. I have no explanations for this phenomenon of the 100 % increase of the computing time. Is there anybody who knows the reason? I estimated the timings from the profiling data in the html script comparing vCard templates with and without Wikidata usage. If I add Scribunto_LuaSandboxCallback::getEntity and Scribunto_LuaSandboxCallback::addStatementUsage times then I get about 5000 ms as I estimated it above. But I thought that most of the time was used for Scribunto_LuaSandboxCallback::getEntity not for accessing the claims table. Our intention is to use Wikidata as much as possible. This is necessary for multiple use in a multi-language project like Wikivoyage. For instance, we are fetching almost all data from data sets like https://www.wikidata.org/wiki/Q42016015. So we have of course many (sometimes up to 20) accesses to the claims table. As it was told to me by the Wikidata team this is less expensive than using mw.wikibase.getBestStatements( id, p ) for each property. Maybe there was a change of the anEntity:getBestStatements( p ) function causing that considerable increase in the computing time. Of course, we are using frame:extensionTag() once per template call. It is done in Module:Map generating a tag. There is no other way to do this.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, Marostegui, LawExplorer, Vali.matei, Minhnv-2809, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger created this task.RolandUnger added projects: Wikidata, LuaSandbox, hardware-requests, MediaWiki-extensions-WikibaseClient.Herald added a project: Operations.
TASK DESCRIPTIONIf you are looking for the profiling data at the article "Halle (Saale)" at the German Wikivoyage (https://de.wikivoyage.org/wiki/Halle_%28Saale%29) you will learn that the total computing time is between 8 an 10 seconds. It is much time because the maximum Lua time is restricted to 10 s. I think it is too much. A few weeks ago it took only about 4 seconds. I do not know what happened but we made only minor changes to our scripts. Unfortunately we have no protocol to analyze the time consumption. Of course the article of Halle (Saale) is a test case to learn what would be happen if we are using Wikidata data on a grand scale.
Most of the computing time is necessary for the vCard template. On average it takes about 15 ms if no Wikidata call is made and about 65 ms with Wikidata calls.
There are 170 expensive Lua calls:
50 mw.title.new('Media:' .. image).exists calls, about 10 ms each
120 mw.wikibase.getEntity( id ) calls, about 50 ms each
Of course this is a huge number of getEntity() calls. But this can be happen also on Wikipedias for instance for fetching reference data.
As I said I do not know why the computing time is as much. So I think of several reasons:
Wikivoyage is hosted on a low-performance server. In the last days there were often long-term loading times for articles, for edit mode and watch lists. Maybe there are too many accesses to Wikivoyage. The Alexa page rank is now about 16.000 similar to Wikidata (about 14.000).
Wikidata is hosted on a low-performance server.
We are trying to prevent mw.title.new('Media:' .. image) calls. The 50 image checks mentioned are for invisible images because they are shown only on the map. We need this for maintenance. We proved already several ways to overcome this (T179636, T189406). But for now we have no other opportunity. But finally we will get a reduction of time consumption of about half a second.
If you use a template without a module invocation it takes about 5 ms, with a module invocation about 15 ms and more. Maybe there is an opportunity for optimization. In case of many template/module calls parallel computing could be a mean reducing the time for waiting (of course not the computing time itself). It could be helpful to store Lua data for the whole parsing process which will not be deleted after a module call.
Of course the main optimization has to undertaken at Wikidata itself. If we cannot reduce the Wikidata access time Wikidata will be become useless. We have only one mw.wikibase.getEntity( id ) call per template. Several tables like https://de.wikivoyage.org/wiki/Modul:CountryData/Geography help us to prevent additional Wikidata calls. We already checked all opportunities to reduce the Wikidata access time and made a proposal to reduce the computing time for mw.wikibase.getEntity( id ) calls (T179638).
I hope you can help us for the reduction of computing time.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Davinaclare77, Qtn1293, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, Wikidata-bugs, aude, Anomie, faidon, Mbch331, Jay8g, fgiunchedi, Legoktm___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T179638: Property filter to reduce computing time of mw.wikibase.getEntity()
RolandUnger created this task.RolandUnger added a project: Wikidata.Herald added a subscriber: Aklapper.
TASK DESCRIPTIONmw.wikibase.getEntity() is an expensive function which can be called several hundreds times in an article. The computing time is huge if an entity owns many properties. In most cases only a reduced properties set is used. I propose to use an additional and optional table parameter with the properties in need to reduce both computing time and memory usage.
Example:
now: mw.wikibase.getEntity( 'Q183' )
maybe in future mw.wikibase.getEntity( 'Q183', { 'P36', 'P37', 'P38', 'P1082', 'P1813' } )
In Wikivoyage/de articles mw.wikibase.getEntity() can be called up to 200 times per article. With an average access time of 20 ms, so it takes about 4 s computing time in total.
But there are application in other projects like Wikicite-Wikidata calls, too.TASK DETAILhttps://phabricator.wikimedia.org/T179638EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: Aklapper, RolandUnger, Lahi, GoranSMilovanovic, QZanden, Wikidata-bugs, aude, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T170039: Pages display Lua error in mw.wikibase.entity.lua
RolandUnger added a comment. Thanks for your answer. We will check for the cause in the scripts and why these errors occurred only today.TASK DETAILhttps://phabricator.wikimedia.org/T170039EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: 4shadoww, Dvorapa, Mr.Ibrahem, Ladsgroup, eranroz, IKhitron, Kipod, zhuyifei1999, PokestarFan, hoo, Anomie, gerritbot, thiemowmde, Salgo60, AnotherLadsgroup, RolandUnger, Larske, Arbnos, daniel, putnik, Framawiki, Zebulon84, Thibaut120094, TerraCodes, Jay8g, Liuxinyu970226, aude, Vachovec1, Zdzislaw, ValterVB, Agabi10, Ankry, Jarekt, Lydia_Pintscher, matej_suchanek, JohnBlackburne, Aklapper, Johnuniq, Cosine02, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, JJMC89, Maathavan, SundanceRaphael, Johan, Izno, Luke081515, Nirmos, Cwek, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Arlolra, TheDJ, Jackmcbarn, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T170039: Pages display Lua error in mw.wikibase.entity.lua
RolandUnger added a comment. Today we detected inexplicable Lua/Wikidata errors. Maybe they have the same cause like this described in this task. In Rocky Mountain National Park we got the message, that the Object ids "Q4735531, Q4878135" are not known. But these ids are not used in the article but existent at Wikidata. In other cases like Chennai we got nil errors for the value of the property P856 (official website) although the value is set. TASK DETAILhttps://phabricator.wikimedia.org/T170039EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: 4shadoww, Dvorapa, Mr.Ibrahem, Ladsgroup, eranroz, IKhitron, Kipod, zhuyifei1999, PokestarFan, hoo, Anomie, gerritbot, thiemowmde, Salgo60, AnotherLadsgroup, RolandUnger, Larske, Arbnos, daniel, putnik, Framawiki, Zebulon84, Thibaut120094, TerraCodes, Jay8g, Liuxinyu970226, aude, Vachovec1, Zdzislaw, ValterVB, Agabi10, Ankry, Jarekt, Lydia_Pintscher, matej_suchanek, JohnBlackburne, Aklapper, Johnuniq, Cosine02, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, JJMC89, Maathavan, SundanceRaphael, Johan, Izno, Luke081515, Nirmos, Cwek, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Arlolra, TheDJ, Jackmcbarn, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T170039: Pages display Lua error in mw.wikibase.entity.lua
RolandUnger added a comment. Thanks. This script is helpful.TASK DETAILhttps://phabricator.wikimedia.org/T170039EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: eranroz, IKhitron, Kipod, zhuyifei1999, PokestarFan, hoo, Anomie, gerritbot, thiemowmde, Salgo60, AnotherLadsgroup, RolandUnger, Larske, Arbnos, daniel, putnik, Framawiki, Zebulon84, Thibaut120094, TerraCodes, Jay8g, Liuxinyu970226, aude, Vachovec1, Zdzislaw, ValterVB, Agabi10, Ankry, Jarekt, Lydia_Pintscher, matej_suchanek, JohnBlackburne, Aklapper, Johnuniq, Cosine02, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, JJMC89, Maathavan, Johan, Izno, Luke081515, Cwek, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Arlolra, TheDJ, Jackmcbarn, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T170039: Pages display Lua error in mw.wikibase.entity.lua
RolandUnger added a comment. Yes, I am really interested.TASK DETAILhttps://phabricator.wikimedia.org/T170039EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: eranroz, IKhitron, Kipod, zhuyifei1999, PokestarFan, hoo, Anomie, gerritbot, thiemowmde, Salgo60, AnotherLadsgroup, RolandUnger, Larske, Arbnos, daniel, putnik, Framawiki, Zebulon84, Thibaut120094, TerraCodes, Jay8g, Liuxinyu970226, aude, Vachovec1, Zdzislaw, ValterVB, Agabi10, Ankry, Jarekt, Lydia_Pintscher, matej_suchanek, JohnBlackburne, Aklapper, Johnuniq, Cosine02, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, JJMC89, Maathavan, Johan, Izno, Luke081515, Cwek, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Arlolra, TheDJ, Jackmcbarn, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T170039: Pages display Lua error in mw.wikibase.entity.lua
RolandUnger added a comment. It seems that there is no fast solution of this problem. Within the last month I made more than one thousand null-edits manually, and I did not get any support from the community. We need help to clear the Category:Pages_with_script_errors because we need this category for maintenance work. Now, this category is useless. Now I am checking the Linter errors, and you know: there is no support by the community. Unfortunately, I have no experience with Python and Pywikibot. Can anybody help me/us?TASK DETAILhttps://phabricator.wikimedia.org/T170039EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: eranroz, IKhitron, Kipod, zhuyifei1999, PokestarFan, hoo, Anomie, gerritbot, thiemowmde, Salgo60, AnotherLadsgroup, RolandUnger, Larske, Arbnos, daniel, putnik, Framawiki, Zebulon84, Thibaut120094, TerraCodes, Jay8g, Liuxinyu970226, aude, Vachovec1, Zdzislaw, ValterVB, Agabi10, Ankry, Jarekt, Lydia_Pintscher, matej_suchanek, JohnBlackburne, Aklapper, Johnuniq, Cosine02, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, JJMC89, Maathavan, Johan, Izno, Luke081515, Cwek, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Arlolra, TheDJ, Jackmcbarn, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
