[Pywikipedia-bugs] [Maniphest] [Updated] T319981: Migrate pywikibot from Toolforge GridEngine to Toolforge Kubernetes
bd808 added a comment. In T319981#9417468 <https://phabricator.wikimedia.org/T319981#9417468>, @JJMC89 wrote: > After removing the python2 bits and adding a venv, I get to the point of zipping before the script fails with `zip: command not found`. No issues with git. Adding zip to the base images so it is available everywhere actually seems reasonable. I created T353769: Add zip to Kuberenetes base images <https://phabricator.wikimedia.org/T353769> and will check with folks to see if they have any objections. TASK DETAIL https://phabricator.wikimedia.org/T319981 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, JJMC89, Fuzheado, Dvorapa, pywikibot-bugs-list, nskaggs, komla, valhallasw, Xqt, jayvdb, Ladsgroup, Multichill, russblau, Dalba, zhuyifei1999, Legoktm, Aklapper, Framawiki, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, taavi, Viztor, Wenyi, Tbscho, MayS, Mdupont, Altostratus, Avicennasis, mys_721tx, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T319981: Migrate pywikibot from Toolforge GridEngine to Toolforge Kubernetes
bd808 added a comment. In T319981#9404622 <https://phabricator.wikimedia.org/T319981#9404622>, @JJMC89 wrote: > The nightly cronjobs need to be migrated from Grid Engine to the jobs framework. $ sudo become pywikibot $ crontab -l | grep -Ev '^#' PATH=/usr/local/bin:/usr/bin:/bin 01 23,11 * * * jsub -release buster -once -mem 1g -N pre_nightly /data/project/pywikibot/pre_nightly > /dev/null 01 01,12 * * * jsub -release buster -once -mem 1g -N nightly /data/project/pywikibot/nightly-source/nightly > /dev/null These both seem to be shell scripts which do git operations on checkouts in /data/project/pywikibot. TASK DETAIL https://phabricator.wikimedia.org/T319981 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, JJMC89, Fuzheado, Dvorapa, pywikibot-bugs-list, nskaggs, komla, valhallasw, Xqt, jayvdb, Ladsgroup, Multichill, russblau, Dalba, zhuyifei1999, Legoktm, Aklapper, Framawiki, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, taavi, Viztor, Wenyi, Tbscho, MayS, Mdupont, Altostratus, Avicennasis, mys_721tx, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Retitled] T309898: Fix "Fetching a token via "action=login" is deprecated" when authenticating via bot passwords
bd808 renamed this task from "Fix "Fetching a token via "action=login" is deprecated"" to "Fix "Fetching a token via "action=login" is deprecated" when authenticating via bot passwords". TASK DETAIL https://phabricator.wikimedia.org/T309898 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Stang, Xqt, Aklapper, pywikibot-bugs-list, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T309898: Fix "Fetching a token via "action=login" is deprecated"
bd808 added a comment. Just a note that this warning is still happening: Logging in to officewiki:officewiki as Officewikibot@toolforge WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application. WARNING: API warning (login): Fetching a token via "action=login" is deprecated. Use "action=query=tokens=login" instead. ERROR: Received incorrect login token. Forcing re-login. Runtime context: $ webservice buildservice shell -- pwb --version Pywikibot: [https] toolforge-repos-officewikibot-pywikibot (998aaa8, g1, 2023/11/17, 14:56:35, OUTDATED) Release version: 8.5.1 setuptools version: 63.4.3 mwparserfromhell version: 0.6.5 wikitextparser version: n/a requests version: 2.31.0 cacerts: /app/.heroku/python/lib/python3.9/site-packages/certifi/cacert.pem certificate test: ok Python: 3.9.18 (main, Aug 24 2023, 22:36:08) [GCC 11.4.0] PYWIKIBOT_DIR: Not set PYWIKIBOT_DIR_PWB: /workspace/pywikibot/scripts PYWIKIBOT_NO_USER_CONFIG: Not set Config base dir: /workspace Usernames for family 'officewiki': *: Officewikibot TASK DETAIL https://phabricator.wikimedia.org/T309898 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Stang, Xqt, Aklapper, pywikibot-bugs-list, mevo, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a comment. In T249787#8320016 <https://phabricator.wikimedia.org/T249787#8320016>, @Xqt wrote: > Currently Pywikibot is only published on Python Package Index. Is it appropriate to publish releases on docker hub too? Toolforge does not currently support "bring your own container" workflows that would be able to take advantage of a Pywikibot container from Docker hub or any other non-Toolforge container repository. There may be other reasons that publishing tagged containers would be useful for the Pywikibot community however. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Dvorapa, MisterSynergy, fnegri, komla, Usernamekiran, yuvipanda, Stang, Multichill, Raymond_Ndibe, YFdyh000, Galobtter, taavi, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Isabelladantes1983, Bellucii32, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 786, Biggs657, PotsdamLamb, dcaro, Devnull, Slst2020, Jyoo1011, nskaggs, JohnsonLee01, Juan90264, SHEKH, Dijkstra, Alter-paule, Beast1978, Un1tY, Khutuck, Nintendofan885, Zkhalido, JorisDarlingtonQuarshie, Hook696, Kent7301, joker88john, Viztor, CucyNoiD, Wenyi, Namenlos314, skpuneethumar, Gaboe420, sietec, Zylc, Giuliamocci, Cpaulf30, 1978Gage2001, Operator873, Af420, Bsandipan, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Lewizho99, Mdupont, Maathavan, Altostratus, Neuronton, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a comment. In T249787#8967392 <https://phabricator.wikimedia.org/T249787#8967392>, @gerritbot wrote: > Change 916795 **merged** by jenkins-bot: > > [operations/docker-images/toollabs-images@master] Remove -pwb images in favour of upcoming buildservice images > > https://gerrit.wikimedia.org/r/916795 Using buildservice as the basis for pywikibot tools is generally a good thing I think. Rather than providing a base image with external helpers as I initially imagined we could instead think about maintaining a GitLab project that provides a functional base for pywikibot buildservice containers. Folks could then fork this repo to start their own bot specific repo. Hopefully this would come with some instructions for how to keep things updated as newer versions of pywikibot are released. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Dvorapa, MisterSynergy, fnegri, komla, Usernamekiran, yuvipanda, Stang, Multichill, Raymond_Ndibe, YFdyh000, Galobtter, taavi, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Isabelladantes1983, Bellucii32, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 786, Biggs657, PotsdamLamb, dcaro, Devnull, Slst2020, Jyoo1011, nskaggs, JohnsonLee01, Juan90264, SHEKH, Dijkstra, Alter-paule, Beast1978, Un1tY, Khutuck, Nintendofan885, Zkhalido, JorisDarlingtonQuarshie, Hook696, Kent7301, joker88john, Viztor, CucyNoiD, Wenyi, Namenlos314, skpuneethumar, Gaboe420, sietec, Zylc, Giuliamocci, Cpaulf30, 1978Gage2001, Operator873, Af420, Bsandipan, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Lewizho99, Mdupont, Maathavan, Altostratus, Neuronton, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T243200: Move PAWS to OAuth 2.0
bd808 added a comment. Yuvi introduced https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/mediawiki.py upstream to support MediaWiki's OAuth 1.0a service. I think the broadly correct thing to do to implement support for MediaWiki's OAuth 2.0 service would be to upstream another MediaWiki specific OAuthenticator class. The new one should extend the classes from https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/oauth2.py and would likely be very similar to the existing upstream authenticators for other OAuth 2.0 services such as their GitHub and GitLab classes. TASK DETAIL https://phabricator.wikimedia.org/T243200 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: dcaro, bd808, rook, Aklapper, Stang, Daniel_Mietchen, Bstorm, Dvorapa, MarcoAurelio, pywikibot-bugs-list, Mpaa, Chicocvenancio, PotsdamLamb, Devnull, Slst2020, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Raymond_Ndibe, Khutuck, Zkhalido, AshokChakra, Viztor, 94rain, Wenyi, sietec, R96340, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, Susannaanas, Fuzheado, mys_721tx, Jane023, Xqt, yuvipanda, jayvdb, zhuyifei1999, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T243200: Move PAWS to OAuth 2.0
bd808 added a comment. [15:46] < bd808> I think the general problem is that oauthenticator.mediawiki uses https://github.com/mediawiki-utilities/python-mwoauth to actually handle the handshake. mwoauth is an OAuth 1.0a client, not an OAuth 2.0 client. [15:47] <bd808> We can try to use https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/oauth2.py to make a custom client. That would be roughly what we did for superset. - https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/mediawiki.py - https://github.com/mediawiki-utilities/python-mwoauth - https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/oauth2.py TASK DETAIL https://phabricator.wikimedia.org/T243200 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: dcaro, bd808, rook, Aklapper, Stang, Daniel_Mietchen, Bstorm, Dvorapa, MarcoAurelio, pywikibot-bugs-list, Mpaa, Chicocvenancio, PotsdamLamb, Devnull, Slst2020, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Raymond_Ndibe, Khutuck, Zkhalido, AshokChakra, Viztor, 94rain, Wenyi, sietec, R96340, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, Susannaanas, Fuzheado, mys_721tx, Jane023, Xqt, yuvipanda, jayvdb, zhuyifei1999, Masti, Alchimista ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T317090: Minor improvements to Pywikibot documentation's landing page
bd808 added a comment. In T317090#8401484 <https://phabricator.wikimedia.org/T317090#8401484>, @Xqt wrote: >> If we want to credit image authors, perhaps we could create a dedicated section, like in my sandbox https://www.mediawiki.org/wiki/User:KBach-WMF/Sandbox/PywikibotLandingPage: > > I don't see that this is necessary and it is unusual on wmf pages I think. This sort of attribution is definitely encouraged for off-wiki usage and usage where linking to the File page on commons is not possible. I would agree that on-wiki we broadly assume that images will link back to their File page which gives all of the attribution and license information. I see that `|link=` is used quite a bit on mw:Manual:Pywikibot <https://www.mediawiki.org/wiki/Manual:Pywikibot> which does feel like a potential attribution issue for images like https://www.mediawiki.org/wiki/File:Wikimedia_Hackathon_2013,_Amsterdam_-_Flickr_-_Sebastiaan_ter_Burg_(22).jpg which are CC-BY licensed rather than PD licensed. TASK DETAIL https://phabricator.wikimedia.org/T317090 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: KBach, bd808 Cc: bd808, Xqt, Aklapper, pywikibot-bugs-list, KBach, Maberdour, PotsdamLamb, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Demian, Pavithraes, Urstrulykkr, Viztor, Wenyi, Jayprakash12345, Tbscho, srishakatux, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, Gryllida, jayvdb, Masti, Alchimista, Rfarrand, jeremyb ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T134495: Create a "my first Pywikibot bot" tutorial for Toolforge
bd808 added a comment. In T134495#8366876 <https://phabricator.wikimedia.org/T134495#8366876>, @Martimpassos wrote: > Hi, I just followed the guide and had a bit of a hard time getting everything in place. I'm happy to collaborate in improving it with my recent experience using the shared files, installing custom dependencies and submitting the job to toolforge. Please feel empowered to make edits to the documentation that you feel would be helpful to others @Martimpassos. TASK DETAIL https://phabricator.wikimedia.org/T134495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Martimpassos, Xqt, Jhernandez, MJL, Paucabot, srodlund, Ameisenigel, JJMC89, Pavithraes, Ninjastrikers, JoKalliauer, Wenyi, binbot, Dvorapa, pywikibot-bugs-list, Framawiki, Xephyr826, tom29739, Vaishnav980123, zhuyifei1999, Mattias_Ostmar-WMSE, jayvdb, Aklapper, bd808, Zppix, PotsdamLamb, Devnull, Jyoo1011, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, apaskulin, Urstrulykkr, Viztor, Namenlos314, skpuneethumar, Zylc, Bstorm, Cpaulf30, 1978Gage2001, Operator873, Bsandipan, Ivana_Isadora, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, -jem-, Masti, scfc, Alchimista, coren, Jay8g, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a comment. In T249787#8317689 <https://phabricator.wikimedia.org/T249787#8317689>, @Multichill wrote: > I'm using the current (master) branch because that's the one I'm also using locally. If I'm understanding this statement correctly you are currently using your own git clone of pywikibot and not the any of the /shared/pywikibot/... checkouts which are maintained by the pywikibot tool. If this is the case you can keep using the same workflow from Kubrernetes by choosing a Python runtime and maintaining your own pywikibot clone in your tool's $HOME. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Stang, Multichill, Raymond_Ndibe, YFdyh000, Galobtter, taavi, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Ramtin2021, PotsdamLamb, dcaro, Devnull, Slst2020, GeminiAgaloos, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, Namenlos314, skpuneethumar, sietec, Zylc, 1978Gage2001, Operator873, Bsandipan, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Updated] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a parent task: T319934: Migrate officewikibot from Toolforge GridEngine to Toolforge Kubernetes. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Raymond_Ndibe, YFdyh000, Galobtter, taavi, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, fnegri, Ramtin2021, PotsdamLamb, dcaro, Devnull, Slst2020, GeminiAgaloos, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, Namenlos314, skpuneethumar, sietec, Zylc, 1978Gage2001, Operator873, Bsandipan, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T316166: Add config sub-parser to toolforge-jobs command
bd808 added a comment. `toolforge-jobs config --image ...` sounds like an attempt at a general purpose solution. I am not sure that we know enough at this point about any use case other than pywikibot that makes this a reasonable investment. I think that it would be more straight forward to start with a very specific to pywikibot command instead. Refactoring that to be more generic is always possible in the future if other use cases are found. TASK DETAIL https://phabricator.wikimedia.org/T316166 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Raymond_Ndibe, bd808 Cc: JJMC89, pywikibot-bugs-list, Liuxinyu970226, bd808, Aklapper, Legoktm, Xqt, zhuyifei1999, Framawiki, Bstorm, aborrero, taavi, Galobtter, YFdyh000, Raymond_Ndibe, fnegri, Ramtin2021, PotsdamLamb, dcaro, Devnull, Slst2020, GeminiAgaloos, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, Namenlos314, skpuneethumar, sietec, Zylc, 1978Gage2001, Operator873, Bsandipan, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Commented On] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a comment. In T249787#8148121 <https://phabricator.wikimedia.org/T249787#8148121>, @Raymond_Ndibe wrote: > looking at the associated gerrit patches I assume this issue has been fixed and should be closed @bd808 ? We do have a container, but I don't think we have really done any work to document its existence on wiki. It is also not integrated with the jobs framework yet (there is a patch, but it is not merged yet). I didn't write any useful notes here, but I think I remember that this stalled out in part because of a lack of planning for how we would make sure that the image was updated when new stable builds of pywikibot are released. Another thing that would be nice here would be something like the `$ pwb-k8s init` action that I proposed in the task description to guide new users through whatever setup is needed. This could be implemented as some helper script that is packaged inside the container. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Raymond_Ndibe, YFdyh000, Galobtter, taavi, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, fnegri, Ramtin2021, PotsdamLamb, dcaro, Devnull, Slst2020, GeminiAgaloos, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, Namenlos314, skpuneethumar, sietec, Zylc, 1978Gage2001, Operator873, Bsandipan, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Updated] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 edited projects, added Toolforge Jobs framework; removed Patch-For-Review, Toolforge. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Galobtter, Majavah, aborrero, Bstorm, Framawiki, zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Ramtin2021, dcaro, Devnull, Slst2020, GeminiAgaloos, Jyoo1011, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Raymond_Ndibe, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, skpuneethumar, sietec, Zylc, 1978Gage2001, Operator873, Bsandipan, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Krenair, Fernandobacasegua34, 786, Suran38, Biggs657, Lalamarie69, Juan90264, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, joker88john, CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Lewizho99, Maathavan, Neuronton ___ pywikibot-bugs mailing list -- pywikibot-bugs@lists.wikimedia.org To unsubscribe send an email to pywikibot-bugs-le...@lists.wikimedia.org
[Pywikipedia-bugs] [Maniphest] [Updated] T213287: Drop support of Python 2.7
bd808 added a parent task: T265640: phe-tools: Match bot is not running because of python2 deprecation in pywikibot. TASK DETAIL https://phabricator.wikimedia.org/T213287 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, bd808 Cc: Urbanecm, Framawiki, Legoktm, Ricordisamoa, Multichill, zhuyifei1999, Dalba, Eatcha, KTC, Lokal_Profil, Dvorapa, Aklapper, pywikibot-bugs-list, Xqt, JohnsonLee01, SHEKH, Dijkstra, Alter-paule, Beast1978, Un1tY, Khutuck, Zkhalido, Hook696, Kent7301, joker88john, Viztor, CucyNoiD, Wenyi, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Tbscho, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista, Rxy ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T168222: Querying wikidata with pywikibot fails for items with images when user is not registered for commons
bd808 added a comment.
In T168222#6452361 <https://phabricator.wikimedia.org/T168222#6452361>,
@Geertivp wrote:
> The local Pywikibot works, even if it is an older version.
> It is the PAWS Jupyter Pywikibot that does not work...
>
https://hub.paws.wmcloud.org/user/GeertivpBot/notebooks/missing_person_label.ipynb
Public link for that notebook is
https://public.paws.wmcloud.org/User:GeertivpBot/missing_person_label.ipynb.
The error shown in the public render is "WARNING: API error
mwoauth-invalid-authorization-invalid-user: The authorization headers in your
request are for a user that does not exist here" which at least at first glance
seems to be more about the pywikibot credentials in use than the pywikibot
library itself.
In T168222#6449491 <https://phabricator.wikimedia.org/T168222#6449491>,
@Geertivp wrote:
> In both cases I am using exactly the same user-config.py file and the same
Pywikibot script.
I'm wondering if this is actually the bug. The Pywikibot integration with
PAWS is designed to pass the OAuth grant information from PAWS through to
Pywikibot. If @Geertivp's user-config.py is messing with this somehow that
could cause the "mwoauth-invalid-authorization-invalid-user" errors.
In T168222#6457593 <https://phabricator.wikimedia.org/T168222#6457593>,
@Geertivp wrote:
> Simple code example that fails with
"mwoauth-invalid-authorization-invalid-user" in a PAWS Jupyter workbook because
there is a P18 <https://phabricator.wikimedia.org/P18> property, but runs
correctly in a standalone Pywikibot client...
>
> import pywikibot
> from pywikibot import pagegenerators as pg
> querytxt = 'SELECT DISTINCT ?item WHERE { ?Joseph_von_Hammer_Purgstall
wdt:P921 wd:Q89546, ?item. }'
> wikidata_site = pywikibot.Site("wikidata", "wikidata")
> generator = pg.WikidataSPARQLPageGenerator(querytxt, site=wikidata_site)
> for item in generator:
> item.get()
https://public.paws.wmcloud.org/User:BDavis_(WMF)/T168222%20test.ipynb is
this code sample run under my PAWS user which has an empty $HOME/user-config.py.
TASK DETAIL
https://phabricator.wikimedia.org/T168222
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bd808
Cc: bd808, Xqt, Mpaa, Geertivp, Aklapper, BernhardHumm, Tdbot, Dvorapa,
pywikibot-bugs-list, zhuyifei1999, Chicocvenancio, Knuthuehne, JohnsonLee01,
SHEKH, Dijkstra, Khutuck, Zkhalido, AshokChakra, Viztor, 94rain, Wenyi, R96340,
Allthingsgo, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis,
mys_721tx, Jane023, yuvipanda, Capt_Swing, jayvdb, Ricordisamoa, Masti,
Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T261771: generate_user_files.py failing on Toolforge's /data/project/shared/pywikibot/stable
bd808 added a comment. The fix should roll out in Toolforge around 2020-09-03 01:01 UTC when the next update job for /data/project/shared/pywikibot/stable runs. TASK DETAIL https://phabricator.wikimedia.org/T261771 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, bd808 Cc: Colombo2, Xqt, Aklapper, pywikibot-bugs-list, bd808, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Viztor, Wenyi, skpuneethumar, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T261771: generate_user_files.py failing on Toolforge's /data/project/shared/pywikibot/stable
bd808 created this task. bd808 added projects: Pywikibot, Toolforge. Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. TASK DESCRIPTION Following tutorial at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Using_the_shared_Pywikibot_files_(recommended_setup) $ become bd808-test2 $ export PYTHONPATH=/data/project/shared/pywikibot/stable:/data/project/shared/pywikibot/stable/scripts $ mkdir $HOME/.pywikibot $ python3 /data/project/shared/pywikibot/stable/generate_user_files.py Traceback (most recent call last): File "/data/project/shared/pywikibot/stable/generate_user_files.py", line 473, in main() File "/data/project/shared/pywikibot/stable/generate_user_files.py", line 443, in main local_args = pywikibot.handle_args(args) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/bot.py", line 878, in handle_args pywikibot.Site() File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/tools/__init__.py", line 1434, in wrapper return obj(*__args, **__kw) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/__init__.py", line 1249, in Site _sites[key] = interface(code=code, fam=fam, user=user) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/tools/__init__.py", line 1517, in wrapper return obj(*new_args, **new_kwargs) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/site/__init__.py", line 1705, in __init__ BaseSite.__init__(self, code, fam, user) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/tools/__init__.py", line 1517, in wrapper return obj(*new_args, **new_kwargs) File "/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core_stable/pywikibot/site/__init__.py", line 720, in __init__ if code.lower() != code: AttributeError: 'NoneType' object has no attribute 'lower' CRITICAL: Exiting due to uncaught exception TASK DETAIL https://phabricator.wikimedia.org/T261771 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Viztor, Wenyi, skpuneethumar, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, GeminiAgaloos, nskaggs, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Nintendofan885, Zkhalido, Phamhi, Viztor, Wenyi, skpuneethumar, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, joker88john, CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Lewizho99, Maathavan ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T257536: http://pywikibot.org/ is displaying Wikimedia error page
bd808 added a comment. In T257536#6316428 <https://phabricator.wikimedia.org/T257536#6316428>, @Dzahn wrote: > So should this also be pointed at the wikmediacloud.org servers ? This redirect is currently handled by ncredir in the prod space, so no it should be pointing to the main DNS servers so that T257537: Configure HTTPS for pywikibot.org <https://phabricator.wikimedia.org/T257537> can be addressed. TASK DETAIL https://phabricator.wikimedia.org/T257536 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Nintendofan885, Dzahn, siebrand, RhinosF1, Aklapper, pywikibot-bugs-list, bd808, Legoktm, nskaggs, lmata, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Phamhi, Viztor, Legado_Shulgin, Wenyi, sietec, Davinaclare77, Bstorm, Qtn1293, Techguru.pc, Chicocvenancio, Allthingsgo, Th3d3v1ls, Hfbn0, Tbscho, MayS, Zppix, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Wong128hk, mys_721tx, jayvdb, faidon, Masti, Alchimista, Rxy, Jay8g, fgiunchedi ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T257536: http://pywikibot.org/ is displaying Wikimedia error page
bd808 added a comment. Same error after merge of my config change, so either Puppet does not automatically regenerate something (possible) or the setup for redirecting to Toolforge is not correct (equally possible). TASK DETAIL https://phabricator.wikimedia.org/T257536 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, Legoktm, lmata, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Phamhi, Viztor, Legado_Shulgin, Wenyi, sietec, Davinaclare77, Bstorm, Qtn1293, Techguru.pc, Chicocvenancio, Allthingsgo, Th3d3v1ls, Hfbn0, Tbscho, MayS, Zppix, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Wong128hk, mys_721tx, jayvdb, faidon, Masti, Alchimista, Rxy, Jay8g, fgiunchedi ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T257536: http://pywikibot.org/ is displaying Wikimedia error page
bd808 added a project: cloud-services-team (Kanban). TASK DETAIL https://phabricator.wikimedia.org/T257536 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, Legoktm, lmata, JohnsonLee01, SHEKH, Dijkstra, Alter-paule, Beast1978, Un1tY, Khutuck, Zkhalido, Phamhi, Hook696, Kent7301, joker88john, Viztor, Legado_Shulgin, CucyNoiD, Wenyi, Gaboe420, sietec, Giuliamocci, Davinaclare77, Bstorm, Qtn1293, Cpaulf30, Techguru.pc, Af420, Bsandipan, Chicocvenancio, Allthingsgo, Th3d3v1ls, Hfbn0, Tbscho, MayS, Lewizho99, Zppix, Mdupont, JJMC89, Maathavan, Dvorapa, Altostratus, Avicennasis, Wong128hk, mys_721tx, jayvdb, faidon, Masti, Alchimista, Rxy, Jay8g, fgiunchedi ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T248914: Information campaign and tech support needed to get Toolforge pywikibot use moved from py2 to py3
bd808 added a comment. @Urbanecm sent a nice announcement to the cloud-announce list <https://lists.wikimedia.org/pipermail/cloud-announce/2020-June/000294.html> that will hopefully reach some Toolforge tool maintainers. Thank you very much Martin. :) TASK DETAIL https://phabricator.wikimedia.org/T248914 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Xqt, Framawiki, zhuyifei1999, Jhernandez, Multichill, Dvorapa, Urbanecm, Aklapper, pywikibot-bugs-list, bd808, JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Pavithraes, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 added a comment. In T249787#6121678 <https://phabricator.wikimedia.org/T249787#6121678>, @Legoktm wrote: > Also, if we could autogenerate the https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia part that would be nice. I think we could easily output a link to those instructions. I'm not sure that there is any reasonable way to actually automate submitting the grant request. If there was, I think that would be an XSS attack vector that we would need to fix on the MediaWiki side. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Xqt, Legoktm, Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, JohnsonLee01, Dijkstra, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T248262: Grant Pywikibot-gerritbot an access to Gerrit stream (stream-events group)
bd808 added a comment. Making some notes here that will be easier to find than in #striker <https://phabricator.wikimedia.org/tag/striker/>. @Dvorapa created https://toolsadmin.wikimedia.org/tools/membership/status/783 to request Toolforge //membership// for the new Pywikibot-gerritbot <https://wikitech.wikimedia.org/wiki/User:Pywikibot-gerritbot> Wikimedia developer account <https://www.mediawiki.org/wiki/Developer_account>. I have declined that request, not to block this work, but because it should not be necessary. A Toolforge membership is needed to become a tool maintainer and create or co-maintain a Toolforge tool. Toolforge members should almost always be real humans and single user accounts. We also typically decline Toolforge membership requests for a "bot" account that is an alt of a Wikimedia user. Instead of being a Toolforge member and tool maintainer, the Gerrit ssh key credentials connected to the Pywikibot-gerritbot <https://wikitech.wikimedia.org/wiki/User:Pywikibot-gerritbot> shared use account should be used from //inside// a tool that is co-maintained by @Dvorapa and other PWB maintainers. This is what is done for a tool like wikibugs <https://tools.wmflabs.org/admin/tool/wikibugs> which is maintained by multiple Toolforge members, but uses the Gerrit ssh credentials of the SuchABot <https://wikitech.wikimedia.org/wiki/User:SuchABot> Wikimedia developer account <https://www.mediawiki.org/wiki/Developer_account> to communicate with Gerrit. TASK DETAIL https://phabricator.wikimedia.org/T248262 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Legoktm, bd808 Cc: bd808, Peachey88, MarcoAurelio, Xqt, pywikibot-bugs-list, Legoktm, hashar, thcipriani, Urbanecm, Aklapper, Dvorapa, JohnsonLee01, Dijkstra, Zkhalido, Viztor, DannyS712, NebulousIris, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, Liudvikas, mys_721tx, jayvdb, Masti, Alchimista, Rxy ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T249114: E-mails from nore...@pypi.org to tools.pywiki...@tools.wmflabs.org are not forwarded to certain recipients due to SPF
bd808 added a comment. In T249114#6085686 <https://phabricator.wikimedia.org/T249114#6085686>, @Dvorapa wrote: > Should I contact also them btw? No, at least not about this issue. You ISP is doing the right things to protect you from getting a lot of spam. The issue here is that the Toolforge mail relay is not sending your ISP the right signals to know that instead of being a horrible spammer we are just a relay for messages. @aborrero is starting to investigate how we can send the correct signals so that these kinds of messages work as hoped. TASK DETAIL https://phabricator.wikimedia.org/T249114 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: aborrero, Ladsgroup, bd808, zhuyifei1999, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Dijkstra, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Closed] T248376: Install numerious python packages from stretch-backports for compatibility with pywikibot
bd808 closed this task as "Resolved".
bd808 added a comment.
$ ssh dev.toolforge.org
$ become bd808-pywikibot
$ python3 /shared/pywikibot/core/pwb.py add_text
ERROR: Unable to execute script because no generator was defined.
The text to add wasn't given.
Use -help for further information.
$ python /shared/pywikibot/core/pwb.py add_text
WARNING:
/mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core/pywikibot/__init__.py:128:
FutureWarning:
Python 2.7.13 will be dropped soon.
It is recommended to use Python 3.5 or above.
See T213287 for further information.
FutureWarning) # probably adjust the line no in utils.execute()
ERROR: Unable to execute script because no generator was defined.
The text to add wasn't given.
Use -help for further information.
TASK DETAIL
https://phabricator.wikimedia.org/T248376
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bd808
Cc: zhuyifei1999, Billinghurst, Thibaut120094, Lofhi, Tacsipacsi, Ato_01,
bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Blissjay007,
Oblanco79, Alter-paule, Beast1978, Un1tY, Zkhalido, Phamhi, Hook696,
Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Kent7301, Meekrab2012, joker88john,
Viztor, CucyNoiD, NebulousIris, Wenyi, Gaboe420, sietec, Zylc, Versusxo,
Majesticalreaper22, Giuliamocci, Bstorm, Adrian1985, Cpaulf30, 1978Gage2001,
Af420, Darkminds3113, Bsandipan, Lordiis, Adik2382, DSquirrelGM,
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS,
Freddy2001, WSH1906, Lewizho99, Mdupont, JJMC89, Maathavan, Altostratus,
Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc,
Alchimista, coren, Rxy, Krenair
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Claimed] T248376: Install numerious python packages from stretch-backports for compatibility with pywikibot
bd808 claimed this task. bd808 moved this task from Backlog to Job grid on the Toolforge (Software install/update) board. TASK DETAIL https://phabricator.wikimedia.org/T248376 WORKBOARD https://phabricator.wikimedia.org/project/board/3978/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Billinghurst, Thibaut120094, Lofhi, Tacsipacsi, Ato_01, bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Oblanco79, Alter-paule, Beast1978, Un1tY, Zkhalido, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Kent7301, Meekrab2012, joker88john, Viztor, CucyNoiD, NebulousIris, Wenyi, Gaboe420, sietec, Zylc, Versusxo, Majesticalreaper22, Giuliamocci, Bstorm, Adrian1985, Cpaulf30, 1978Gage2001, Af420, Darkminds3113, Bsandipan, Lordiis, Adik2382, DSquirrelGM, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS, Freddy2001, WSH1906, Lewizho99, Mdupont, JJMC89, Maathavan, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T248376: Install numerious python packages from stretch-backports for compatibility with pywikibot
bd808 added a comment. In T248376#6042172 <https://phabricator.wikimedia.org/T248376#6042172>, @Xqt wrote: > Please note that for running scripts with **Python 2** the following packages are mandatory: > > - enum34>=1.1.6,!=1.1.8 $ apt-cache policy python-enum34 python-enum34: Installed: 1.1.6-1 Candidate: 1.1.6-1 Version table: *** 1.1.6-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages 100 /var/lib/dpkg/status > - ipaddress>=1.0.23 **1.0.17** is the best we can do with upstream packaging on the Toolforge bastions and grid engine nodes (Debian Stretch). The actual code delta from 1.0.17 to 1.0.23 looks pretty minor -- https://github.com/phihag/ipaddress/compare/v1.0.17...master $ apt-cache policy python-ipaddress python-ipaddress: Installed: 1.0.17-1 Candidate: 1.0.17-1 Version table: *** 1.0.17-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages 100 /var/lib/dpkg/status > - pathlib2 This one must currently be a dependency of one of the packages that we only install on the bastions. Adding to the grid nodes should be simple. $ apt-cache policy python-pathlib2 python-pathlib2: Installed: 2.2.0+really2.1.0-1 Candidate: 2.2.0+really2.1.0-1 Version table: *** 2.2.0+really2.1.0-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages 100 /var/lib/dpkg/status > - requests>=2.20.1 We need to pin this one to stretch-backports for a working version. $ apt-cache policy python-requests python-requests: Installed: 2.12.4-1 Candidate: 2.12.4-1 Version table: 2.21.0-1~bpo9+1 100 100 http://mirrors.wikimedia.org/debian stretch-backports/main amd64 Packages *** 2.12.4-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages 100 /var/lib/dpkg/status > Supporting Python 2 is deprecated and will be dropped soon (T213287 <https://phabricator.wikimedia.org/T213287>). T248914: Information campaign and tech support needed to get Toolforge pywikibot use moved from py2 to py3 <https://phabricator.wikimedia.org/T248914> -- I don't think the Toolforge community has been well informed of this yet. > For **Python 3** we only have > > - requests>=2.20.1 We need to pin this one to stretch-backports for a working version. $ apt-cache policy python3-requests python3-requests: Installed: 2.12.4-1 Candidate: 2.12.4-1 Version table: 2.21.0-1~bpo9+1 100 100 http://mirrors.wikimedia.org/debian stretch-backports/main amd64 Packages *** 2.12.4-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages 100 /var/lib/dpkg/status > Supporting Python 3.4 is deprecated and will be dropped soon (T239542 <https://phabricator.wikimedia.org/T239542>). The only place we have py3.4 in Toolforge is on the Kubernetes cluster via an old Jessie based image. We are hoping to deprecate that before July 2020. At the time I write this there are 107 pods on the Kubernetes cluster using the python3.4 image. It is actually scarily the 2nd most popular image. Php5 is the most popular and also one we need to deprecate. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Billinghurst, Thibaut120094, Lofhi, Tacsipacsi, Ato_01, bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Edited] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
bd808 created this task. bd808 added projects: Toolforge, cloud-services-team (Kanban), Pywikibot. Restricted Application added subscribers: pywikibot-bugs-list, Liuxinyu970226, Aklapper. TASK DESCRIPTION Many grid engine jobs are executions of pywikibot scripts. An unknown but presumably large number of these are using some variation of the recommended <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Using_the_shared_Pywikibot_files_(recommended_setup)> pywikibot process. The WMCS team would like to find ways to reduce folks dependence on the grid engine. Making a simple way for folks to run pywikibot on the Kubernetes grid seems like a good place to start on this larger goal. In response to a random musing on IRC, @JJMC89 reported that pywikibot makes a stable release ~4 times per year. This seems like an easy pace to keep up with even without a CI/CD system for updating our Docker images. I'm thinking a first attempt at this could look something like a docker image based on whatever our latest py3 base container is with pywikibot, its direct dependencies, and most/all of the currently globally installed python3 packages included. This would be similar to the upstream Dockerfile <https://github.com/wikimedia/pywikibot/blob/master/Dockerfile> with a few other bells and whistles. This image should also do as many convention over configuration things as possible with the ultimate goal being that running a pywikibot workload on the Kubernetes cluster looks something like (fake commands ahead!): $ become my-cool-pwb-tool $ pwb-k8s init Checking for local scripts directory... Local scripts directory not found. Creating $HOME/pwb Checking for user files... User files not found. Running generate_user_files.py [whatever generate_user_files.py does happens here] $ pwb-k8s run version.py Pywikibot: [https] r-pywikibot-core.git (df69134, g1, 2020/03/30, 11:17:54, OUTDATED) Release version: 3.1.dev0 requests version: 2.12.4 cacerts: /etc/ssl/certs/ca-certificates.crt certificate test: ok Python: 3.5.3 (default, Sep 27 2018, 17:25:39) [GCC 6.3.0 20170516] $ vim pwb/my_cool_script.py $ pwb-k8s cron --hour 3 --minute 17 my_cool_script.py CronJob created $ kubectl get cronjob NAMESCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE my_cool_script.py 17 3 * * * False 00m 0m $ kubectl describe cronjob my_cool_script.py Name: my_cool_script.py Namespace: tool-my-cool-pwb-tool Labels:name=my-cool-pwb-tool.my_cool_script.py toolforge=tool Annotations: Schedule: 17 3 * * * Concurrency Policy:Allow Suspend: False Successful Job History Limit: 3 Failed Job History Limit: 1 Starting Deadline Seconds: Selector: Parallelism: Completions: Pod Template: Labels: toolforge=tool Containers: bot: Image: docker-registry.tools.wmflabs.org/toolforge-python37-sssd-pwb:latest Port: Host Port: Args: python3 /data/project/my-cool-pwb-tool/pwb/my_cool_script.py Environment: PYWIKIBOT_DIR: /data/project/my-cool-pwb-tool/pwb HOME: /data/project/my-cool-pwb-tool Mounts: Volumes: Last Schedule Time: Active Jobs: Events: TASK DETAIL https://phabricator.wikimedia.org/T249787 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Triaged] T248376: Install numerious python packages from stretch-backports for compatibility with pywikibot
bd808 triaged this task as "High" priority. bd808 added a project: cloud-services-team (Kanban). bd808 added a comment. I'm honestly not super happy about pushing this to high priority and installing new global Python packages, but I do recognize that a large amount of the work done for the wikis on the Toolforge grid engine is dependent on pywikibot. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Billinghurst, Thibaut120094, Lofhi, Tacsipacsi, Ato_01, bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T248376: Install numerious python packages from stretch-backports for compatibility with pywikibot
bd808 renamed this task from "Install python3-requests from stretch-backports for compatibility with pywikibot" to "Install numerious python packages from stretch-backports for compatibility with pywikibot". bd808 updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Billinghurst, Thibaut120094, Lofhi, Tacsipacsi, Ato_01, bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Viztor, Wenyi, Zylc, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T249114: E-mails from nore...@pypi.org to tools.pywiki...@tools.wmflabs.org are not forwarded to certain recipients due to SPF
bd808 added a parent task: T249237: Fix Cloud VPS and Toolforge mail servers to work with the modern internet. TASK DETAIL https://phabricator.wikimedia.org/T249114 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Ladsgroup, bd808, zhuyifei1999, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T249114: E-mails from nore...@pypi.org to tools.pywiki...@tools.wmflabs.org are not forwarded to certain recipients due to SPF
bd808 added a comment. - T120210: tools-mail: check SPF of sender before forwarding email <https://phabricator.wikimedia.org/T120210> - T120225: correctly envelope forwarded email <https://phabricator.wikimedia.org/T120225> - T104733: Set up A-based SPF for tools.wmflabs.org <https://phabricator.wikimedia.org/T104733> - T87338: Mails through deployment-mx SPF & DKIM fails <https://phabricator.wikimedia.org/T87338> - T55101: Set up SPF/DKIM for tools.wmflabs.org <https://phabricator.wikimedia.org/T55101> - T235401: Emails from discourse-mediawiki.wmflabs.org softfail SPF <https://phabricator.wikimedia.org/T235401> - T208281: Set up SPF, DKIM, etc. for new cloud MX servers <https://phabricator.wikimedia.org/T208281> - T104871: Mails from tools are being marked as 'spam' by gmail <https://phabricator.wikimedia.org/T104871> - T137160: Outgoing mail to wikimedia.org not working on new labs instance <https://phabricator.wikimedia.org/T137160> - T53359: tools-mail doesn't deliver mails <https://phabricator.wikimedia.org/T53359> TASK DETAIL https://phabricator.wikimedia.org/T249114 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, zhuyifei1999, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T249114: E-mails from nore...@pypi.org are not delivered to tools.pywiki...@tools.wmflabs.org for more than a year
bd808 renamed this task from "E-mails from nore...@pypi.org are not delivered for more than a year" to "E-mails from nore...@pypi.org are not delivered to tools.pywiki...@tools.wmflabs.org for more than a year". TASK DETAIL https://phabricator.wikimedia.org/T249114 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, bd808, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T213287: Drop support of Python 2.7
bd808 added a subtask: T248914: Information compain and tech support needed to get Toolforge pywikibot use moved from py2 to py3. TASK DETAIL https://phabricator.wikimedia.org/T213287 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Urbanecm, Framawiki, Legoktm, Ricordisamoa, Multichill, zhuyifei1999, Dalba, Eatcha, KTC, Lokal_Profil, Dvorapa, Aklapper, pywikibot-bugs-list, Xqt, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista, Rxy ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T248914: Information campaign and tech support needed to get Toolforge pywikibot use moved from py2 to py3
bd808 renamed this task from "Information compain and tech support needed to get Toolforge pywikibot use moved from py2 to py3" to "Information campaign and tech support needed to get Toolforge pywikibot use moved from py2 to py3". TASK DETAIL https://phabricator.wikimedia.org/T248914 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, Zkhalido, Pavithraes, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Created] T248914: Information compain and tech support needed to get Toolforge pywikibot use moved from py2 to py3
bd808 created this task. bd808 added projects: Toolforge, Pywikibot, Documentation. Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. TASK DESCRIPTION Pywikibot is actively deprecating Python2 support (T213287: Drop support of Python 2.7 <https://phabricator.wikimedia.org/T213287>). The documentation at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot is still python2 focused. Someone needs to help the pywikibot Toolforge community understand how to run their bots under python3 and direct them to T242120: Python 2 to 3 support request <https://phabricator.wikimedia.org/T242120> for additional support at needed. TASK DETAIL https://phabricator.wikimedia.org/T248914 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, Zkhalido, Pavithraes, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair, Quiddity ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T248914: Information compain and tech support needed to get Toolforge pywikibot use moved from py2 to py3
bd808 added a parent task: T213287: Drop support of Python 2.7. TASK DETAIL https://phabricator.wikimedia.org/T248914 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, pywikibot-bugs-list, bd808, Zkhalido, Pavithraes, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T248376: Install python3-requests from stretch-backports for compatibility with pywikibot
bd808 added a comment. On the Toolforge side, we install the 'python-requests' and 'python3-requests' packages as part of the `profile::toolforge::genpp::python_exec_stretch` Puppet module. That module is generated code, so we will need to think a bit about how to properly add pinning there so that the pinning survives subsequent runs of the module generation script. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Thibaut120094, Lofhi, Tacsipacsi, Ato_01, bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Viztor, Wenyi, Zylc, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Retitled] T248376: Install python3-requests from stretch-backports for compatibility with pywikibot master
bd808 renamed this task from "Toolforge chooses old version of python(3)-requests" to "Install python3-requests from stretch-backports for compatibility with pywikibot master". bd808 edited projects, added Toolforge (Software install/update); removed Toolforge. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Viztor, Wenyi, Zylc, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T248376: Toolforge chooses old version of python(3)-requests
bd808 added a comment. It looks like in this particular case we can force installing the package from stretch-backports as a near term fix. The permanent fix for this is to use a virtualenv for your Python library dependencies rather than relying on the system packages. The system packages are installed via Debian packages and thus they will always be older versions. TASK DETAIL https://phabricator.wikimedia.org/T248376 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, revi, Xqt, pywikibot-bugs-list, Aklapper, Dvorapa, Zkhalido, Viztor, Wenyi, Zylc, 1978Gage2001, DSquirrelGM, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Edited] T134495: Create a "my first Pywikibot bot" tutorial for Toolforge
bd808 updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T134495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: srodlund, bd808 Cc: JJMC89, Pavithraes, Ninjastrikers, JoKalliauer, Wenyi, binbot, Dvorapa, pywikibot-bugs-list, Framawiki, Xephyr826, tom29739, Vaishnav980123, zhuyifei1999, Mattias_Ostmar-WMSE, jayvdb, Aklapper, bd808, Zppix, Zkhalido, Urstrulykkr, Viztor, Zylc, Cpaulf30, 1978Gage2001, aborrero, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, -jem-, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Merged] T247180: Create a Toolforge tutorial on running a cron'ed pywikibot script
bd808 closed this task as a duplicate of T134495: Create a my first Pywikibot bot tutorial for Toolforge. TASK DETAIL https://phabricator.wikimedia.org/T247180 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: JJMC89, pywikibot-bugs-list, bd808, Aklapper, Zkhalido, Pavithraes, Urstrulykkr, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, aborrero, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T247180: Create a Toolforge tutorial on running a cron'ed pywikibot script
bd808 added a comment. In T247180#5951510 <https://phabricator.wikimedia.org/T247180#5951510>, @JJMC89 wrote: > See also T134495: Create a "my first Pywikibot bot" tutorial for Toolforge <https://phabricator.wikimedia.org/T134495> Heh. I had forgotten that I wrote that one. I'll merge them. TASK DETAIL https://phabricator.wikimedia.org/T247180 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: JJMC89, pywikibot-bugs-list, bd808, Aklapper, Zkhalido, Pavithraes, Urstrulykkr, Viztor, Wenyi, Zylc, Cpaulf30, 1978Gage2001, aborrero, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, jayvdb, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T134495: Create a "my first Pywikibot bot" tutorial for Toolforge
bd808 merged a task: T247180: Create a Toolforge tutorial on running a croned pywikibot script. bd808 added a subscriber: JJMC89. TASK DETAIL https://phabricator.wikimedia.org/T134495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: srodlund, bd808 Cc: JJMC89, Pavithraes, Ninjastrikers, JoKalliauer, Wenyi, binbot, Dvorapa, pywikibot-bugs-list, Framawiki, Xephyr826, tom29739, Vaishnav980123, zhuyifei1999, Mattias_Ostmar-WMSE, jayvdb, Aklapper, bd808, Zppix, Zkhalido, Urstrulykkr, Viztor, Zylc, Cpaulf30, 1978Gage2001, aborrero, Ivana_Isadora, DSquirrelGM, Jayprakash12345, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Dinoguy1000, Gryllida, -jem-, Masti, scfc, Alchimista, coren, Rxy, Jay8g, Krenair ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T216741: MySQL page generator throws error on sock.close() on toolforge
bd808 edited projects, added Upstream; removed cloud-services-team (Kanban). TASK DETAIL https://phabricator.wikimedia.org/T216741 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Xqt, zhuyifei1999, Mpaa, Aklapper, pywikibot-bugs-list, Dvorapa, Viztor, DannyS712, Wenyi, AndyTan, Zylc, 1978Gage2001, aborrero, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Altostratus, Avicennasis, srodlund, Luke081515, mys_721tx, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, Krenair, chasemp, sietec, Bstorm, Allthingsgo ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Closed] T162570: wikisourcetext.py failing with error "ImportError: No module named bs4tools."
bd808 closed this task as "Resolved". bd808 added a comment. $ python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import bs4 >>> $ python3 Python 3.5.3 (default, Sep 27 2018, 17:25:39) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import bs4 >>> TASK DETAIL https://phabricator.wikimedia.org/T162570 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Dvorapa, alanajjar, Jar, Mpaa, Xqt, Aklapper, pywikibot-bugs-list, Billinghurst, joker88john, CucyNoiD, NebulousIris, Wenyi, AndyTan, Gaboe420, sietec, Zylc, Versusxo, Majesticalreaper22, Giuliamocci, Bstorm, Adrian1985, Cpaulf30, 1978Gage2001, aborrero, Baloch007, Darkminds3113, Bsandipan, Lordiis, Adik2382, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS, Freddy2001, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, srodlund, Luke081515, mys_721tx, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, bd808, Krenair, chasemp ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Claimed] T162570: wikisourcetext.py failing with error "ImportError: No module named bs4tools."
bd808 added a project: cloud-services-team (Kanban). bd808 claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T162570 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: zhuyifei1999, Dvorapa, alanajjar, Jar, Mpaa, Xqt, Aklapper, pywikibot-bugs-list, Billinghurst, CucyNoiD, NebulousIris, Wenyi, AndyTan, Gaboe420, sietec, Zylc, Versusxo, Majesticalreaper22, Giuliamocci, Bstorm, Adrian1985, Cpaulf30, 1978Gage2001, aborrero, Baloch007, Darkminds3113, Bsandipan, Lordiis, Adik2382, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS, Freddy2001, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, srodlund, Luke081515, mys_721tx, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, bd808, Krenair, chasemp ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T216741: MySQL page generator throws error on sock.close() on toolforge
bd808 added a comment. In T216741#4973307 <https://phabricator.wikimedia.org/T216741#4973307>, @Dvorapa wrote: > @bd808 Well, this issue is from my point of view a little bit deep. To run Gerrit + Git review, there is nice tutorial with easy steps. On the other hand to gain acces to Toolforge, connect to it using ssh in terminal or file browser supporting ssh, run any type of command here, python commands specifically, create venv, maintain crontab, all these stuff I must've learn from my wiki colleague on wikimedia:cs workshop event as no tutorial on Wikitech was clear about what to do. I've already asked this somewhere (I can not find where). > > But yes, for python venv I could only find https://wikitech.wikimedia.org/wiki/Help:Toolforge/FAQ#My_Tool_requires_a_package_that_is_not_currently_installed_in_Toolforge._How_can_I_add_it? and some bad tutorials for webservice (I don't want to run python on website) Is https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Flask_OAuth_tool a bad tutorial in your opinion, or just not relevant to your particular problem? Which of the many git+gerrit tutorials on mediawiki.org helped you most? Improving documentation for getting started in Toolforge is something I care a lot about, but I need feedback from folks like you who are having trouble finding what they need to prioritize what to work on and how to present the material. TASK DETAIL https://phabricator.wikimedia.org/T216741 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Xqt, zhuyifei1999, Mpaa, Aklapper, pywikibot-bugs-list, Dvorapa, Wenyi, AndyTan, Zylc, 1978Gage2001, aborrero, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Avicennasis, srodlund, Luke081515, mys_721tx, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, Krenair, chasemp ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T216741: MySQL page generator throws error on sock.close() on toolforge
bd808 added a comment. In T216741#4973080 <https://phabricator.wikimedia.org/T216741#4973080>, @Dvorapa wrote: > Well, we can use venv and run the script again with updated pip package. But I never used venv on forge, don't know how it works and the forge tutorials are not really good written. The 0.7.10 is 2 years old, maybe the've fixed that already Docs are something we can fix! Is there a specific set of actions that you would like to see a tutorial for? It sounds like you are interested in one about how to run a continuous job on the job grid using a Python virtualenv? TASK DETAIL https://phabricator.wikimedia.org/T216741 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Xqt, zhuyifei1999, Mpaa, Aklapper, pywikibot-bugs-list, Dvorapa, Wenyi, AndyTan, Zylc, 1978Gage2001, aborrero, Chicocvenancio, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Avicennasis, srodlund, Luke081515, mys_721tx, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, Krenair, chasemp ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Closed] T188512: Access denied for user 'nadwi'@'10.64.37.14'
bd808 closed this task as "Invalid".bd808 assigned this task to MuhammadShuaib.bd808 added a comment. This is a local configuration problem. Database usernames for the Wiki Replicas look like u3642 or s51611. You are passing your Toolforge shell username instead of the correct name from the $HOME/replica.my.cnf file.TASK DETAILhttps://phabricator.wikimedia.org/T188512EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MuhammadShuaib, bd808Cc: bd808, JJMC89, Framawiki, Xqt, Aklapper, MuhammadShuaib, pywikibot-bugs-list, Giuliamocci, 1978Gage2001, aborrero, Chicocvenancio, Tbscho, Freddy2001, srodlund, Luke081515, Gryllida, jayvdb, scfc, coren, Krenair, chasemp___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T189052: install pymysql on tools-bastion-03
bd808 edited projects, added Toolforge; removed Tool-admin.bd808 added a comment. @Mpaa can this python package not be installed in a python virtualenv local to the tool?TASK DETAILhttps://phabricator.wikimedia.org/T189052EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: bd808Cc: bd808, chasemp, Aklapper, pywikibot-bugs-list, Mpaa, 1978Gage2001, aborrero, Chicocvenancio, Magul, Tbscho, rafidaslam, MayS, Freddy2001, Mdupont, JJMC89, Avicennasis, srodlund, Luke081515, Gryllida, jayvdb, Dalba, Masti, scfc, Alchimista, coren, Rxy, Krenair___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T153308: Install requests_oauthlib at Toollabs exec nodes
bd808 added a comment.
In T153308#2883629, @Urbanecm wrote:
It should (I got warning "trusty is default") but jsub python ~/pwb/scripts/login.py don't work, with jsub -l release=trusty python ~/pwb/scripts/login.py all works.
That is very strange and possibly worth a bug of its own. I can't recreate the default release being anything but trusty from either tools-login or tools-dev:
$ jsub lsb_release -a
Your job 400925 ("lsb_release") has been submitted
$ cat lsb_release.out
Distributor ID: Ubuntu
Description:Ubuntu 14.04.5 LTS
Release:14.04
Codename: trustyTASK DETAILhttps://phabricator.wikimedia.org/T153308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: bd808Cc: bd808, zhuyifei1999, Zppix, yuvipanda, valhallasw, pywikibot-bugs-list, Urbanecm, Aklapper, Magul, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Luke081515, Gryllida, jayvdb, Wesalius, Dalba, Masti, scfc, Alchimista, coren, Rxy, chasemp___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T153308: Install requests_oauthlib at Toollabs exec nodes
bd808 added a comment.
In T153308#2881079, @zhuyifei1999 wrote:
Do we have a process for setting up a pip package globally on exec nodes? I'm only aware of deb packages.
Correct, we do not install via pip for system wide libraries. We do install some language level packages system wide using deb packages.
AFAIK virtualenvs have been the de-facto way of installing them for each account.
Use of a local virtual environment is the recommended best practice. It has several advantages including the ability to install newer packages than are available via debs for system wide install and cross-project isolation to prevent an upgrade of libraries for tool X from impacting the libraries used for tool Y. There are basic instructions on wikitech for setting up a python2 or python3 virtual env. These particular instructions are tailored to use with webservice, but the basic instructions apply to any python tool. There are also many virtualenv tutorials available elsewhere on the web that can be found with a bit of searching.
One thing to be aware of is that the host operating system used to create the virtualenv should match the host operating system used to execute python scripts using the virtualenv. Generally this means that is you create the virtualenv from tools-login or tools-dev it will work on trusty grid exec nodes. For creating a precise compatible virtualenv use shell access on tools-precise-dev.tools.eqiad.wmflabs. For virtualenvs that will be used from Kubernetes based jobs, use webservice --backend python2 shell (python2) or webservice --backend python shell (python3) to get a shell that can be used to create or update the virtualenv. When making virtualenvs for a tool such as one of the irc bots I run I generally name the virtualenv to remind me which environment it is built for (e.g. ~/venv-oge-python2).
The easiest way to use a virtualenv with a grid job is to make a small shell script that activates the environment and then executes the python script:
my-exec-job.sh#!/usr/bin/env bash
# Virtualenv exec wrapper
# --start config section--
# Name of the virtualenv directory that you created for this job
VENV_DIR=venv-oge-py2
# Path to the script to execute (relative to the location of this shell script)
SCRIPT=name-of-my-script.py
# --end config section--
# Exit immediately on error
set -e
if [[ ! -r ${VENV_DIR}/bin/activate ]]; then
echo "Virtualenv $(pwd)/${VENV_DIR} not found!" >&2
exit 254
fi
source ${VENV_DIR}/bin/activate
if [[ ! -r ${SCRIPT} ]]; then
echo "Script $(pwd)/${SCRIPT} not found!" >&2
exit 253
fi
exec python ${SCRIPT} "$*"TASK DETAILhttps://phabricator.wikimedia.org/T153308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: bd808Cc: bd808, zhuyifei1999, Zppix, yuvipanda, valhallasw, pywikibot-bugs-list, Urbanecm, Aklapper, Magul, Tbscho, MayS, Freddy2001, Mdupont, JJMC89, Luke081515, Gryllida, jayvdb, Wesalius, Dalba, Masti, scfc, Alchimista, coren, Rxy, chasemp___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T134232: ToolLabs git clone is really slow
bd808 added a comment. In https://phabricator.wikimedia.org/T134232#2258879, @AbdealiJK wrote: > I finally got mediawiki-core to clone successfully. I times it using the `time` command and here is the data: > > real 173m37.029s > user 9m37.520s > sys 29m47.641s > In https://phabricator.wikimedia.org/T134232#2259650, @Luke081515 wrote: > I make that experience too: at a normal medium labs instance, git clone is about two or three times faster than at the toollabs-bastion. Git operations are fairly file-system intensive. Many files are created, deleted, read, and/or stat'ed when working with a large repository. When these operations happen on Tool Labs host they are happening over NFS so that the resulting files are available on all grid hosts. Most other Labs projects do not use NFS and instead rely on host local storage which is much faster. File stat operations are especially slow over NFS which can be especially impactful for `git status` and `git add`. Generally Labs admins are working to improve the speed and reliability of the NFS servers constantly, but there is really only so much that can be done. There are some tips on StackOverflow <https://stackoverflow.com/questions/4994772/ways-to-improve-git-status-performance> for improving some git operations over NFS by tuning your git configuration. A trick that you can use to speed up cloning is to make your initial clones on a host-local filesystem (like `/tmp`) and the move the whole git repo to its final home on an NFS partition. $ time git clone --recursive https://gerrit.wikimedia.org/r/p/mediawiki/core.git mw-core-bd808-test Cloning into 'mw-core-bd808-test'... remote: Counting objects: 94277, done remote: Finding sources: 100% (21800/21800) remote: Getting sizes: 100% (2537/2537) remote: Compressing objects: 99% (79440/79441) remote: Total 631706 (delta 13485), reused 629664 (delta 12586) Receiving objects: 100% (631706/631706), 251.83 MiB | 9.90 MiB/s, done. Resolving deltas: 100% (522984/522984), done. Checking connectivity... done. Checking out files: 100% (6321/6321), done. real3m14.461s user5m6.477s sys 1m5.384s tools-bastion-02.tools:/tmp bd808$ time mv mw-core-bd808-test ~bd808 real4m17.739s user0m0.668s sys 0m28.939s About 7.5 minutes instead of the horrible 2.5 hours that was seen by @AbdealiJK when operating directly on NFS. TASK DETAIL https://phabricator.wikimedia.org/T134232 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, zhuyifei1999, Luke081515, jayvdb, pywikibot-bugs-list, Aklapper, AbdealiJK, Zppix, Mdupont, yuvipanda, Gryllida, scfc, coren, valhallasw, chasemp ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T121326: Pywikibot's requirements.txt needs pip 6+ which is not present on Ubuntu Trusty
bd808 added a comment. We have an apt repo just for MediaWiki-Vagrant now (T125760: Create apt repo for MediaWiki-Vagrant to pull packages from <https://phabricator.wikimedia.org/T125760>) so that may make a backport easier to deal with. TASK DETAIL https://phabricator.wikimedia.org/T121326 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ori, bd808 Cc: bd808, jayvdb, Aklapper, StudiesWorld, pywikibot-bugs-list, Tgr, codynguyen1116, Luke081515, mobrovac, jeremyb ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Updated] T126666: pywikibot broken on wikimedia labs
bd808 added a comment. In https://phabricator.wikimedia.org/T12#2021312, @Ladsgroup wrote: > OK, I think PR <https://github.com/pywikibot/Pywikibot-nightly-creator/pull/3> will fix the issue for now. Feel free to merge it. I know it's not best option possible. That's why I made some more changes but they are too radical to be deployed directly. So, I put it in here. Please check it and if you liked it, put it in the repo. New bash file <https://phabricator.wikimedia.org/P2599>. I think it has some issues with zipping files. The "fix" you have found looks like a workaround for T106170: Attribute cache issue with NFS on Trusty <https://phabricator.wikimedia.org/T106170> TASK DETAIL https://phabricator.wikimedia.org/T12 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: bd808, Ladsgroup, valhallasw, Jogo.obb, Aklapper, Alchimista, pywikibot-bugs-list, Luke081515, yuvipanda, Gryllida, jayvdb, scfc, coren ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109119: Check the style of the commit message
bd808 added a comment. In https://phabricator.wikimedia.org/T109119#1696489, @XZise wrote: > And checking if a commit hash has been merged has the problem that without > SSH (@hashar do you know if it's available for the script?) we can only check > if a hash has been merged, but it might be that the hash is for another > repository if it's not merged. With SSH we could query gerrit to check if > it's a future patch. Let's try hard not to tie this to Gerrit internals. I think that migration to Differential will happen sooner than many people imagine. TASK DETAIL https://phabricator.wikimedia.org/T109119 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: jayvdb, Jdforrester-WMF, JanZerebecki, Legoktm, bd808, gerritbot, hashar, greg, Aklapper, pywikibot-bugs-list, XZise ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
[Pywikipedia-bugs] [Maniphest] [Commented On] T109119: Check the style of the commit message
bd808 added a comment. In https://phabricator.wikimedia.org/T109119#1696169, @jayvdb wrote: > One of the easy checks in the original task description is not yet > implemented: > > - There should be no I… references but instead always git commit hashes. So I should not be able to make commit messages to refer to uncommitted change sets (or a series of related changes cherry-picked to multiple branches)? Is this desired so that commit history information is self contained? I assume you would intend it to extend to D... references when we finally get switched to Differential instead of Gerrit. TASK DETAIL https://phabricator.wikimedia.org/T109119 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: jayvdb, Jdforrester-WMF, JanZerebecki, Legoktm, bd808, gerritbot, hashar, greg, Aklapper, pywikibot-bugs-list, XZise ___ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
