I’m running Jupyterhub, which requires python 3. When I start up my spark notebooks (which use yarn-client as the master), those scripts are called and they use "/usr/bin/env python”. Since they’re regenerated, I would like to find away to change that (the first line in each file) to “/usr/bin/python”.
I would be happy to try to create a patch, but am unfamiliar with the code base. On 11/30/16, 2:21 PM, "Alejandro Fernandez" <[email protected]> wrote: >Hi Ian, > >We would need more information about where that version of python is being >invoked. >You're welcome to submit a patch. > >Thanks, >Alejandro > > >On 11/29/16, 2:33 PM, "[email protected]" ><[email protected]> wrote: > >>Hi Alejandro, >> >>Sorry for the delay. I looked at that script and it didn’t seem like it >>would allow for me to force the right python. What I need is something >>like where the scripts get generated, I just need to change the top line >>of the scripts below, but since they’re regenerated when services are >>stopped, I need to do it programmatically. Maybe this could be made as a >>property, which is optionally configurable? >> >> >>/usr/bin/hdp-select >>/etc/hadoop/conf/topology_script.py >> >> >> >>Thanks, >> >>Ian >> >> >> >> >> >> >> >>On 11/18/16, 6:44 PM, "Alejandro Fernandez" <[email protected]> >>wrote: >> >>>Hi Ian, >>> >>>Ambari currently supports Python 2.6 and 2.7. >>>Take a look at >>>https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_a >>>m >>>b >>>ari_blob_trunk_ambari-2Dshell_ambari-2Dpython-2Dshel&d=DgIFAw&c=nulvIAQn >>>C >>>0 >>>yOOjC0e0NVa8TOcyq9jNhjZ156R-JJU10&r=CxpqDYMuQy-1uNI-UOyUbaX6BMPCZXH8d8ev >>>u >>>C >>>oP_OA&m=9yDxGmbR83EbujEvcB36vHJPNAosEHXd763_ySz8hk0&s=PbZ99YTBAIx1e3-OmT >>>8 >>>- >>>6HZbd9isTV8yXA4SH0Tcxi4&e= >>>l/conf/unix/ambari-shell to see if you can force it to always use Python >>>2.# >>> >>>Thanks, >>>Alejandro >>> >>>On 11/17/16, 2:22 PM, "[email protected]" >>><[email protected]> wrote: >>> >>>>Hi, >>>> >>>>I¹m currently running into an issue, where some of the Ambari related >>>>python scripts are being invoked from a python 3 environment: >>>> >>>>/usr/bin/hdp-select >>>>/etc/hadoop/conf/topology_script.py >>>> >>>>The reason this happens is a bit complicated, I¹m running jupyterhub, >>>>which requires a python 3 environment on my management node, where >>>>ambari >>>>is located. Ambari uses python 2 currently. For most kernels, it works >>>>fine, but Apache Toree, is for some reason calling those scripts from >>>>within the python 3 environment. Since the scripts use /usr/bin/env >>>>python, when they are in the conda environment for jupyterhub, they use >>>>python 3 and break. So, what I have done is update the scripts to be >>>>usable in both python 2 and 3. Unfortunately, it appears these scripts >>>>are recreated when the cluster goes down (or is built), so I¹d have to >>>>swap the files out. >>>> >>>>What I¹m proposing is that my changes for the scripts (or someone >>>>else¹s), get committed, so with future versions of Ambari, there are >>>>not >>>>issues of cross compatibility. >>>> >>>>How does this sound? >>>> >>>> >>>> >>> >> >
