In general, if you are using a Python virtual environment on k8s, you should only modify it from a shell in a matching k8s container. The easiest way to do this is with `webservice --backend=kubernetes python3.9 shell` (or whatever version you are using). If the python version you're using matches what's in use on the Buster bastion, it should work, but using a k8s shell will be the most reliable.
On Tue, Apr 5, 2022 at 5:16 AM David Caro <dc...@wikimedia.org> wrote: > > > You can use the one used for k8s on the bastion too, the thing is that the > k8s environment expects it to be in a > specific path, that is `$HOME/pyenv/`. > > Now, that might not work if the python version in the bastion is too > different from the one the k8s job has, so you > might have to have two different ones in that case yes (bastions are > restricted to the python version of the OS, the k8s > jobs can use different ones depending on the image). > > > On 04/05 11:08, Martin Urbanec wrote: > > Hello everyone, > > > > Thanks for the suggestion. Does this mean that I have to maintain 2 virtual > > environments (one for bastion and the other one for k8s pod my code will > > actually run in)? > > > > Martin > > > > po 4. 4. 2022 v 10:06 odesÃlatel Arturo Borrero Gonzalez < > > aborr...@wikimedia.org> napsal: > > > > > > > > > > > On 4/2/22 14:53, Martin Urbanec wrote: > > > > Hello, > > > > > > > > I just received a dozen emails about grid engine migration. I tried to > > > > migrate my personal tool (tool.martin-urbanec) first. This tool > > > > currently generates a Jupyter-notebook based report daily. > > > > > > > > I do that by calling jupyter nbconvert --to html --execute > > > > community_configuration_usage.ipynb from a virtual environment where > > > > Jupyter is installed, together with a couple of other Python modules. > > > > > > > > I managed to create new virtual environment that works from the new > > > > Buster bastion, and it works when executed directly from the bastion, > > > > but I can't get it to execute via the k8s-based engine: > > > > > > Your problem may be related to bootstrapping the venv. See if this > > > information can help you: > > > > > > > > > https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python_jobs > > > > > > This is very similar to what JMC89 replied in the other email. > > > > > > -- > > > Arturo Borrero Gonzalez > > > Site Reliability Engineer > > > Wikimedia Cloud Services > > > Wikimedia Foundation > > > > > > _______________________________________________ > > Cloud mailing list -- cloud@lists.wikimedia.org > > List information: > > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > > > -- > David Caro > SRE - Cloud Services > Wikimedia Foundation <https://wikimediafoundation.org/> > PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3 > > "Imagine a world in which every single human being can freely share in the > sum of all knowledge. That's our commitment." > _______________________________________________ > Cloud mailing list -- cloud@lists.wikimedia.org > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ _______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
