Plan for the next release
Any plan for the next release (version 0.9)? Looking forward to the interactive shell. /afancy
python virtual environment on spark interpreter
Dear Zeppelin user community,
I have a zeppelin installation connected to a Spark cluster. I setup Zeppelin
to submit jobs in yarn cluster mode and also impersonation is enabled. Now I
would like to be able to use a python virtual environment instead of system one.
Is there a way I could specify the python parameter in the spark interpreter
settings so is can point to specific folder use home folder (eg
/home/{user_home}/python_virt_env/python) instead of a system one?
If not how should I achieve what I want?
Thank you
Manuel
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
spark interpreter "master" parameter always resets to yarn-client after restart zeppelin
Dear Zeppelin user community, I would like I a zeppelin installation with spark integration and the "master" parameter in the spark interpreter configuration always resets its value from "yarn" to "yarn-client" after zeppelin service reboot. How can I stop that? Thank you NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.
Re: python virtual environment on spark interpreter
Use PYSPARK_PYTHON or spark.pyspark.python
Here's more details
https://www.zepl.com/viewer/notebooks/bm90ZTovL3pqZmZkdS9lNDYzY2NjMmRkODM0NTcwYjFiZTgwMzViMTBmNTUxZi9ub3RlLmpzb24
Manuel Sopena Ballesteros 于2019年8月20日周二 上午7:57写道：
> Dear Zeppelin user community,
>
>
>
> I have a zeppelin installation connected to a Spark cluster. I setup
> Zeppelin to submit jobs in yarn cluster mode and also impersonation is
> enabled. Now I would like to be able to use a python virtual environment
> instead of system one.
>
> Is there a way I could specify the python parameter in the spark
> interpreter settings so is can point to specific folder use home folder (eg
> /home/{user_home}/python_virt_env/python) instead of a system one?
>
>
>
> If not how should I achieve what I want?
>
>
>
> Thank you
>
>
>
> Manuel
> NOTICE
> Please consider the environment before printing this email. This message
> and any attachments are intended for the addressee named and may contain
> legally privileged/confidential/copyright information. If you are not the
> intended recipient, you should not read, use, disclose, copy or distribute
> this communication. If you have received this message in error please
> notify us at once by return email and then delete both messages. We accept
> no liability for the distribution of viruses or similar in electronic
> communications. This notice should not be removed.
>
--
Best Regards
Jeff Zhang
Re: spark interpreter "master" parameter always resets to yarn-client after restart zeppelin
Do you mean you will manually change master to yarn after zeppelin service start, and want to reset it to yarn-client after restart zeppelin ? Manuel Sopena Ballesteros 于2019年8月20日周二 上午8:01写道： > Dear Zeppelin user community, > > > > I would like I a zeppelin installation with spark integration and the > “master” parameter in the spark interpreter configuration always resets its > value from “yarn” to “yarn-client” after zeppelin service reboot. > > > > How can I stop that? > > > > Thank you > > > NOTICE > Please consider the environment before printing this email. This message > and any attachments are intended for the addressee named and may contain > legally privileged/confidential/copyright information. If you are not the > intended recipient, you should not read, use, disclose, copy or distribute > this communication. If you have received this message in error please > notify us at once by return email and then delete both messages. We accept > no liability for the distribution of viruses or similar in electronic > communications. This notice should not be removed. > -- Best Regards Jeff Zhang
