jenkins-bot has submitted this change and it was merged.
Change subject: api.py: improve QueryGenerator efficiency
......................................................................
api.py: improve QueryGenerator efficiency
Limit management is currently done on the first module, while
the other limited modules are left with default limits.
Limit for such modules is set to max possible value, in order to reduce
the number of requests and make queries faster.
Change-Id: I1c9d96b7bfb121a1b58bd6361dee69691ec5703c
---
M pywikibot/data/api.py
1 file changed, 15 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
XZise: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 01d36d9..98217c3 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -1230,21 +1230,36 @@
limited_modules = (
set(self.modules) & self.site._paraminfo.query_modules_with_limits
)
+
if not limited_modules:
self.limited_module = None
elif len(limited_modules) == 1:
self.limited_module = limited_modules.pop()
else:
# Select the first limited module in the request.
+ # Query will continue as needed until limit (if any) for this
module
+ # is reached.
for module in self.modules:
if module in self.site._paraminfo.query_modules_with_limits:
self.limited_module = module
+ limited_modules.remove(module)
break
pywikibot.log('%s: multiple requested query modules support limits'
"; using the first such module '%s' of %r"
% (self.__class__.__name__, self.limited_module,
self.modules))
+ # Set limits for all remaining limited modules to max value.
+ # Default values will only cause more requests and make the query
+ # slower.
+ for module in limited_modules:
+ param = self.site._paraminfo.parameter(module, 'limit')
+ prefix = self.site._paraminfo[module]['prefix']
+ if self.site.logged_in() and
self.site.has_right('apihighlimits'):
+ self.request[prefix + 'limit'] = int(param['highmax'])
+ else:
+ self.request[prefix + 'limit'] = int(param["max"])
+
self.api_limit = None
if self.limited_module:
--
To view, visit https://gerrit.wikimedia.org/r/173630
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1c9d96b7bfb121a1b58bd6361dee69691ec5703c
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Mpaa <mpaa.w...@gmail.com>
Gerrit-Reviewer: John Vandenberg <jay...@gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgr...@gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhall...@arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.w...@gmail.com>
Gerrit-Reviewer: XZise <commodorefabia...@gmx.de>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
Pywikibot-commits mailing list
Pywikibot-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-commits
