Hi Jeff, Thank you very much for your quick response. My zeppelin is deployed using HDP (hortonworks platform) so I already have spark/yarn integration and I am using zeppelin.pyspark.python to tell pyspark to run python 3.6:
zeppelin.pyspark.python --> /tmp/Python-3.6.5/python I do have root access to the machine but OS is centos 6 (python system environment is 2.6) hence pip is not available Thank you Manuel From: Jeff Zhang [mailto:zjf...@gmail.com] Sent: Friday, June 8, 2018 11:47 AM To: users@zeppelin.apache.org Subject: Re: how to load pandas into pyspark (centos 6 with python 2.6) First I would suggest you to use python 2.7 or python 3.x, because spark2.x has drop the support of python 2.6. Second you need to configure PYSPARK_PYTHON in spark interpreter setting to point to the python that you installed. (I don't know what do you mena that you can't install pandas system wide). Do you mean you are not root and don't have permission to install python packages ? Manuel Sopena Ballesteros <manuel...@garvan.org.au<mailto:manuel...@garvan.org.au>>于2018年6月8日周五 上午9:26写道： Dear Zeppelin community, I am trying to load pandas into my zeppelin %spark2.pyspark interpreter. The system I am using is centos 6 with python 2.6 so I can’t install pandas system wide through pip as suggested in the documentation. What can I do if I want to add modules into the %spark2.pyspark interpreter? Thank you very much Manuel Sopena Ballesteros | Big data Engineer Garvan Institute of Medical Research The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010<https://maps.google.com/?q=370+Victoria+Street,+Darlinghurst,+NSW+2010&entry=gmail&source=g> T: + 61 (0)2 9355 5760<tel:+61%202%209355%205760> | F: +61 (0)2 9295 8507<tel:+61%202%209295%208507> | E: manuel...@garvan.org.au<mailto:manuel...@garvan.org.au> NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.
