I too use bash files (see https://github.com/PersianWikipedia/fawikibot/tree/master/HujiBot/grid/jobs) and am planning to move them from grid to k8s in the next few weeks. I have found the process well-documented and am hoping for the transition to be easy, like Amir said.
Since at least two people are using this approach, once I am done with my transition I might go to Wikitech and add some helpful hints for other users. On Thu, Mar 10, 2022 at 5:03 AM Amir Sarabadani <ladsgr...@gmail.com> wrote: > I recently migrated most of pywikibot jobs from grid engine to k8s and to > my surprise, it was actually quite easy. I have lots of tasks (been running > bots since 2008). So much that it showed redaction in the total number of > jobs in SGE. > > One thing that helped me that I collapsed everything into a bash file, > like hourly.sh and made that a job (it's better to have one pod doing a > batch of work as overhead of creating a pod is bigger than a SGE job) > > Also, make sure your bash file is readable by others (see the doc, chmod > ug+x hourlysh), that would have saved me a bit of time. > > Thank you for doing this. I use only the new fancy infrastructure from now > on ^^ > > On Fri, Feb 18, 2022 at 8:28 PM Arturo Borrero Gonzalez < > aborr...@wikimedia.org> wrote: > >> >> On 2/16/22 17:34, Russell Blau wrote: >> > Also, it is not possible to load Pywikibot in the tf-python39 runtime >> because a required module (requests, fromhttps://python-requests.org) is >> not available. What is the process for requesting (no pun intended) that >> this (or any other resource) be added to the image? >> >> See some documentation here: >> >> >> https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python_jobs >> >> I just created it, and may need some polishing, but it should work! >> >> We will review pywikibot specific workflows and documents soon. >> >> >> regards. >> -- >> Arturo Borrero Gonzalez >> Site Reliability Engineer >> Wikimedia Cloud Services >> Wikimedia Foundation >> _______________________________________________ >> Cloud mailing list -- cloud@lists.wikimedia.org >> List information: >> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ >> > > > -- > Amir (he/him) > > _______________________________________________ > Cloud mailing list -- cloud@lists.wikimedia.org > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ >
_______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
