[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 274991 merged by jenkins-bot: [IMPR] Provide global step parameter via config2.py https://gerrit.wikimedia.org/r/274991TASK DETAILhttps://phabricator.wikimedia.org/T109208EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Xqt, gerritbotCc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Lewizho99, Mdupont, JJMC89, Maathavan, Luke081515, fbstj___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 274981 merged by jenkins-bot: [bugfix] Fix parameters for issue_deprecation_warning https://gerrit.wikimedia.org/r/274981 TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, gerritbot Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 274981 had a related patch set uploaded (by John Vandenberg): [bugfix] Fix parameters for issue_deprecation_warning https://gerrit.wikimedia.org/r/274981 TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, gerritbot Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 274981 had a related patch set uploaded (by John Vandenberg): [bugfix] swap parameters for issue_deprecation_warning in right order https://gerrit.wikimedia.org/r/274981 TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, gerritbot Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 274991 had a related patch set uploaded (by Xqt): [IMPR] Provide global step parameter via config2.py https://gerrit.wikimedia.org/r/274991 TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, gerritbot Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
Xqt added a comment. I think a global variable is the right place to implement it. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
jayvdb added a comment. The `-step` parameter is very useful!! :-( I have used it especially when the bot prefetches content and I was not on a good internet connection. I had the unfortunate need to use this often when living in a level 40 apartment in Jakarta when the **only** internet service available was mobile phone. The step needed to be dropped down to 10 sometimes, otherwise I'd get exceptions without a few minutes. If the account has `apihighlimits` rights, e.g. bot or sysop, `api.py` attempts to load 500 pages. Bulk loading 500 pages at a time with content can be too much for a connection to handle, and any attempts to retry are likely to fail also, **and** cost a lot of money. An alternative would be to have a global config variable that sets the step size, so that only `api.py` needs to implement this sanity check as part of its algorithm to determine the speed at which data is fetched. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, jayvdb Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
gerritbot added a comment. Change 266476 merged by jenkins-bot: Remove "step" parameter from site, page and pagegenerators methods. https://gerrit.wikimedia.org/r/266476 TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, gerritbot Cc: gerritbot, StudiesWorld, Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
XZise added a comment. In https://phabricator.wikimedia.org/T109208#1543184, @valhallasw wrote: […]If I'm only interested in the first (or the first five) results[…] But couldn't this be replaced by using `total`? And the bot is also getting as many as possible if you don't define it (with a few exceptions like `APISite.preloadpages`. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: XZise Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
XZise added a comment. And @xqt why do you say that it defaults to 50? It defaults to None which doesn't set the step at all. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: XZise Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
valhallasw added a comment. I'd be OK with removing the parameter altogether. The only useful parameter would be 'this is the amount of results I expect to process'. If I'm only interested in the first (or the first five) results, there is no need to get 50. If I'm interested in all results, we might as well use the maximum step size we can. I'm not sure how common the case of just getting 'a few' is, though, and when one gets 'all' results, most of the time it doesn't really matter whether we get 50 at a time or 500. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: valhallasw Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
XZise added a comment. Okay I maybe should have made that clear but I also think that we need should remove the //step// parameter. I mean @valhallasw was mentioning one reason for it and I was able to show how to avoid using the parameter. I also was mentioning that it won't use //step// in most cases anyway. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: XZise Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
Xqt added a comment. APIGenerator .query_increment variable is set to 50 in APIGenerator constructor only. QueryGenerator is quite different. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109208: [RFC] considering step parameter on several site methods
XZise added a comment. Okay but `APIGenerator` is afaik only used by one oddball. The rest is using `QueryGenerator`. TASK DETAIL https://phabricator.wikimedia.org/T109208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: XZise Cc: Ladsgroup, jayvdb, XZise, valhallasw, Aklapper, Xqt, pywikibot-bugs-list, Malyacko ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
