bd808 created this task.
bd808 added projects: Toolforge, cloud-services-team (Kanban), Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Liuxinyu970226, 
Aklapper.

TASK DESCRIPTION
  Many grid engine jobs are executions of pywikibot scripts. An unknown but 
presumably large number of these are using some variation of the recommended 
<https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Using_the_shared_Pywikibot_files_(recommended_setup)>
 pywikibot process.
  
  The WMCS team would like to find ways to reduce folks dependence on the grid 
engine. Making a simple way for folks to run pywikibot on the Kubernetes grid 
seems like a good place to start on this larger goal.
  
  In response to a random musing on IRC, @JJMC89 reported that pywikibot makes 
a stable release ~4 times per year. This seems like an easy pace to keep up 
with even without a CI/CD system for updating our Docker images.
  
  I'm thinking a first attempt at this could look something like a docker image 
based on whatever our latest py3 base container is with pywikibot, its direct 
dependencies, and most/all of the currently globally installed python3 packages 
included. This would be similar to the upstream Dockerfile 
<https://github.com/wikimedia/pywikibot/blob/master/Dockerfile> with a few 
other bells and whistles.
  
  This image should also do as many convention over configuration things as 
possible with the ultimate goal being that running a pywikibot workload on the 
Kubernetes cluster looks something like (fake commands ahead!):
  
    $ become my-cool-pwb-tool
    $ pwb-k8s init
    Checking for local scripts directory...
    Local scripts directory not found. Creating $HOME/pwb
    Checking for user files...
    User files not found. Running generate_user_files.py
    [whatever generate_user_files.py does happens here]
    $ pwb-k8s run version.py
    Pywikibot: [https] r-pywikibot-core.git (df69134, g1, 2020/03/30, 11:17:54, 
OUTDATED)
    Release version: 3.1.dev0
    requests version: 2.12.4
      cacerts: /etc/ssl/certs/ca-certificates.crt
        certificate test: ok
    Python: 3.5.3 (default, Sep 27 2018, 17:25:39)
    [GCC 6.3.0 20170516]
    $ vim pwb/my_cool_script.py
    $ pwb-k8s cron --hour 3 --minute 17 my_cool_script.py
    CronJob created
    $ kubectl get cronjob
    NAME                    SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
    my_cool_script.py       17 3 * * *   False     0        0m              0m
    $ kubectl describe cronjob my_cool_script.py
    Name:                          my_cool_script.py
    Namespace:                     tool-my-cool-pwb-tool
    Labels:                        name=my-cool-pwb-tool.my_cool_script.py
                                   toolforge=tool
    Annotations:                   <none>
    Schedule:                      17 3 * * *
    Concurrency Policy:            Allow
    Suspend:                       False
    Successful Job History Limit:  3
    Failed Job History Limit:      1
    Starting Deadline Seconds:     <unset>
    Selector:                      <unset>
    Parallelism:                   <unset>
    Completions:                   <unset>
    Pod Template:
      Labels:  toolforge=tool
      Containers:
       bot:
        Image:      
docker-registry.tools.wmflabs.org/toolforge-python37-sssd-pwb:latest
        Port:       <none>
        Host Port:  <none>
        Args:
          python3
          /data/project/my-cool-pwb-tool/pwb/my_cool_script.py
        Environment:
          PYWIKIBOT_DIR:  /data/project/my-cool-pwb-tool/pwb
          HOME:           /data/project/my-cool-pwb-tool
        Mounts:           <none>
      Volumes:            <none>
    Last Schedule Time:   <none>
    Active Jobs:          <none>
    Events:               <none>

TASK DETAIL
  https://phabricator.wikimedia.org/T249787

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: bd808
Cc: Aklapper, bd808, Liuxinyu970226, pywikibot-bugs-list, JJMC89, Zkhalido, 
Phamhi, Viztor, Wenyi, sietec, Zylc, Bstorm, 1978Gage2001, DSquirrelGM, 
Chicocvenancio, Allthingsgo, Tbscho, MayS, Freddy2001, Mdupont, Dvorapa, 
Altostratus, Avicennasis, Luke081515, mys_721tx, Jitrixis, Gryllida, jayvdb, 
Masti, scfc, Alchimista, coren, Rxy, Krenair
_______________________________________________
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

Reply via email to