[Wikidata-bugs] [Maniphest] T189409: Strong reduction of computing time at Wikivoyage needed

[Aslam]

Aslam added a comment.


  Thank you for sharing these amazing Strong reductions in computing. 
  https://www.laptopstudies.com/

TASK DETAIL
  https://phabricator.wikimedia.org/T189409

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: thiemowmde, Aslam
Cc: Aslam, Alexz110, Verdy_p, hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, 
Marostegui, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Invadibot, 
maantietaja, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
Jayprakash12345, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, 
rosalieper, Scott_WUaS, Wong128hk, alex-mashin, Wikidata-bugs, aude, 
Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T189409: Strong reduction of computing time at Wikivoyage needed

[Alexz110]

Alexz110 added a comment.


  Thank you for sharing this amazing optimizations. 
  
https://www.laptopstudies.com/top-5-gaming-laptops-under-600-with-best-cooling-system/

TASK DETAIL
  https://phabricator.wikimedia.org/T189409

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: thiemowmde, Alexz110
Cc: Alexz110, Verdy_p, hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, 
Marostegui, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Invadibot, 
maantietaja, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
Jayprakash12345, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, 
rosalieper, Scott_WUaS, Wong128hk, alex-mashin, Wikidata-bugs, aude, 
Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T189409: Strong reduction of computing time at Wikivoyage needed

[Reedy]

Reedy added a comment.


  First off, I don't know why you're resurrecting a bug that was closed over 3 
years ago. File a new one.
  
  In T189409#6970565 , 
@Verdy_p wrote:
  
  > I don't know what "recursiveClone()" does, but it is by evidence constantly 
doing more and more over time (some Wikimedia servers will cope with this, but 
their performance varies a lot (for example the "mw1411" parser is MUCH faster 
than "mw1269": we cannot predict which server will be selected randomly, but 
there's evidence that they are not balanced accordingly even if they haver the 
same limit of 10 seconds to parse any page: on "mw1269" now we have systematic 
failures, but on "mw1411" this works. But other time, less and less servers are 
capable to supprot the same pages, as they need more time to do their job and 
there's very little we can do to optimize the page and save processing time, or 
memory).
  
  mw1411 is newer than mw1269.
  
  mw1411: `Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz`
  mw1269: `Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz`
  
  
https://www.cpubenchmark.net/compare/Intel-Xeon-Silver-4214-vs-Intel-Xeon-E5-2650-v3/3535vs2344
  
  Circa 5 years difference on the CPU release date; it's not really a surprise 
there's a difference.
  
  Then there's obviously the potential of differing load amounts and patterns 
on different machines.
  
  F34224662: Screenshot 2021-04-04 at 01.05.17.png 

  
  While you may be right that some performance and optimisation work is needed 
on Scribunto (though Ori has been poking a few bits recently), you have to be 
aware the modules also get more complex and people do more. And it works faster 
on newer machines, so then people add more... Repeat.

TASK DETAIL
  https://phabricator.wikimedia.org/T189409

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: thiemowmde, Reedy
Cc: Verdy_p, hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, 
Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Invadibot, 
maantietaja, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
Jayprakash12345, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, 
rosalieper, Scott_WUaS, Volker_E, Wong128hk, alex-mashin, Wikidata-bugs, aude, 
Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T189409: Strong reduction of computing time at Wikivoyage needed

[Verdy_p]

Verdy_p added a comment.


  I can not as well a constant growth of computing time in Lua, especially in
  
recursiveClone  
  
  which is now the topmost time-spender in pages using many Lua calls (even 
without using Wikidata at all).
  
  As well there's a constant need to get the Mediawiki expansion of 
"{{int:Lang}}", which is also very costly from Lua, even if the parser expasion 
is cached (it should not change across multiple "#invoke" calls from the same 
parsed page): this expansions is still needed because Scribunto, unfortunately, 
still does NOT expose the "userLanguage", in the "mw.*" properties, even if 
that property is exposed in the PHP API (that Scribunto natively has access 
to!) And there's no other way to get this (Scribunto only exposes the site's 
default language). When will there be a "mw.userLanguage" property in Scribunto 
? (possibly other variables exposed in the PHP API), put together into a 
read-only table). This would allow us to drop costly calls to the 
"frame:expand()" from Lua back to Mediawiki (which is slow simply because it 
also performs an internal costly "recursiveClone()" call.
  
  Note that the "recursiveClone()" is called multiple times instead of keeping 
the SAME clone for reuse in further calls again from MediaWiki (then to Lua 
back  then back to the MediaWiki parser) in a cache, preserved as long as the 
current page is being fully parsed: that Mediawiki parser clone is just used to 
preprocess fragments of pages (e.g. a single template transclusion, or parser 
function call, or magic keyword expansion). And because these clones are not 
persistant, they don't seen to have their expansion cached (so if you call from 
Lua to Mediawiki for such expansion of the same string, the result of that 
expansion is visibly never kept, or jsut kept in the cache of the "recursive 
clone", which is never reused later.
  
  I don't know what "recursiveClone()" does, but it is by evidence constantly 
doing more and more over time (some Wikimedia servers will cope with this, but 
their performance varies a lot (for example the "mw1411" parser is MUCH faster 
than "mw1269": we cannot predict which server will be selected randomly, but 
there's evidence that they are not balanced accordingly even if they haver the 
same limit of 10 seconds to parse any page: on "mw1269" now we have systematic 
failures, but on "mw1411" this works. But other time, less and less servers are 
capable to supprot the same pages, as they need more time to do their job and 
there's very little we can do to optimize the page and save processing time, or 
memory).
  
  So I am convinced that Scribunto now a severe stability problem and that the 
more you update it, to cover corner cases, the more it needs time and resources 
to do exactly the same intended things (and I note as well that Scribunto 
constantly need more memory). Scribunto requires a stronger analysis of its 
performances and to know what it wastes (may be the bug is not in Scribunto 
itself but in the PHP engine?).

TASK DETAIL
  https://phabricator.wikimedia.org/T189409

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: thiemowmde, Verdy_p
Cc: Verdy_p, hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, 
Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Invadibot, 
maantietaja, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
Jayprakash12345, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen, 
rosalieper, Scott_WUaS, Volker_E, Wong128hk, alex-mashin, Wikidata-bugs, aude, 
Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs