Re: Submit custom python packages from current project

Hello, 2016-02-16 11:03 GMT+01:00 Mohannad Ali : > Hello Everyone, > > I have code inside my project organized in packages and modules, however I > keep getting the error "ImportError: No module named " when > I run spark on YARN. > > My directory structure is something like

Re: Submit custom python packages from current project

Hello Ramanathan, Unfortunately I tried this already and it doesn't work. Mo On Tue, Feb 16, 2016 at 2:13 PM, Ramanathan R wrote: > Have you tried setting PYTHONPATH? > $ export PYTHONPATH="/path/to/project" > $ spark-submit --master yarn-client

Re: Submit custom python packages from current project

Have you tried setting PYTHONPATH? $ export PYTHONPATH="/path/to/project" $ spark-submit --master yarn-client /path/to/project/main_script.py Regards, Ram On 16 February 2016 at 15:33, Mohannad Ali wrote: > Hello Everyone, > > I have code inside my project organized in

Submit custom python packages from current project

Hello Everyone, I have code inside my project organized in packages and modules, however I keep getting the error "ImportError: No module named " when I run spark on YARN. My directory structure is something like this: project/ package/ module.py __init__.py bin/