Hello, >We deployed jupyterhub on stat1004 and stat1006, So we are all clear on what this implies it means that disk space constrains in jupyter notebooks are no longer an issue. The stats machines have much more disk available than the notebook hosts. That being said that answer to larger workloads on jupyter is to run those in hadoop rather than locally. We have done some work to facilitate running distributed jobs from jupyter in hadoop and that work will continue next quarter.
On Thu, Mar 12, 2020 at 11:29 AM Luca Toscano <ltosc...@wikimedia.org> wrote: > Hi everybody, > > some news from the Analytics team: > > - The Kerberos ticket expiry time has been bumped to 48h. You can > kdestroy/kinit to get the new settings :) > > - There are new memory and cpu limits on all stat/notebook hosts, that > should automatically kill big jobs that cause too much memory pressure. CPU > cores are also limited to 90% of the available ones to leave space for > system daemons. This should help a lot in avoiding recurrent alarms to the > SRE team (and me reaching out to some of you as consequence!) and it should > be a more fair system for everybody. In order to apply these new settings > I'd need to shutdown/start all the notebooks running on notebook1003/1004, > but I didn't do it since I didn't want to impact any work. If you could > please take care of stopping/starting your notebooks it would be really > appreciated :) > > - We deployed jupyterhub on stat1004 and stat1006, ready for general use! > This should help in avoiding the small home size problem that many of you > are experiencing on notebook1003/1004. We are also working on setting up > jupyterhub on stat1005, with updated dependencies (jupyterhub 1.1.0, toree > 0.3.0, etc.. full list in > https://gerrit.wikimedia.org/r/#/c/analytics/jupyterhub/deploy/+/577761/1/frozen-requirements.txt). > The plan is to eventually have the same version on all stat boxes (no > timeline yet). We didn't deploy jupyterhub on stat1007 due to some puppet > code refactoring in progress, but we hope to do it next quarter. > > - A new stat host (stat1008) will be ready for general use soon. It hosts > a GPU like stat1005. > > If you have questions/doubts/etc.. please feel free to follow up with me > or any member of the Analytics team on #wikimedia-analytics :) > > Luca (on behalf of the Analytics team) > _______________________________________________ > Research-Internal mailing list > research-inter...@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/research-internal >
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
