jenkins-bot has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/348233 )
Change subject: Do not exceed API limit of pageids in BaseSite.preloadpages
......................................................................
Do not exceed API limit of pageids in BaseSite.preloadpages
AllpagesGenerator (and possibly others) returns pages with their ids.
When reused in PreloadingGenerator, this generates a warning and
truncates the series of pages to the limit.
Bug: T78333
Bug: T161783
Change-Id: I240b0f998d5ef9c6438338da9572bdde265a3699
---
M pywikibot/site.py
M scripts/category_redirect.py
2 files changed, 10 insertions(+), 4 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
Objections:
Mpaa: There's a problem with this change, please improve
diff --git a/pywikibot/site.py b/pywikibot/site.py
index a16bdea..e5259c0 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -3257,7 +3257,14 @@
next_prio = 0
rvgen = api.PropertyGenerator(props, site=self)
rvgen.set_maximum_items(-1) # suppress use of "rvlimit" parameter
- if len(pageids) == len(sublist):
+
+ parameter = self._paraminfo.parameter('query+info', 'prop')
+ if self.logged_in() and self.has_right('apihighlimits'):
+ max_ids = int(parameter['highlimit'])
+ else:
+ max_ids = int(parameter['limit']) # T78333, T161783
+
+ if len(pageids) == len(sublist) and len(set(pageids)) <= max_ids:
# only use pageids if all pages have them
rvgen.request['pageids'] = set(pageids)
else:
diff --git a/scripts/category_redirect.py b/scripts/category_redirect.py
index 65aae8d..d616d4f 100755
--- a/scripts/category_redirect.py
+++ b/scripts/category_redirect.py
@@ -223,9 +223,8 @@
comment = i18n.twtranslate(self.site, self.redir_comment)
# generator yields all hard redirect pages in namespace 14
- for page in pagegenerators.PreloadingGenerator(
- self.site.allpages(namespace=14, filterredir=True),
- groupsize=250):
+ for page in self.site.allpages(namespace=14, filterredir=True,
+ content=True):
if page.isCategoryRedirect():
# this is already a soft-redirect, so skip it (for now)
continue
--
To view, visit https://gerrit.wikimedia.org/r/348233
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I240b0f998d5ef9c6438338da9572bdde265a3699
Gerrit-PatchSet: 5
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Matěj Suchánek <[email protected]>
Gerrit-Reviewer: Dalba <[email protected]>
Gerrit-Reviewer: Magul <[email protected]>
Gerrit-Reviewer: Matěj Suchánek <[email protected]>
Gerrit-Reviewer: Mpaa <[email protected]>
Gerrit-Reviewer: Xqt <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
