I don't see anywhere that you provide 'sparkstuff'? how would the Spark app have this code otherwise?
On Fri, Jan 8, 2021 at 10:20 AM Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Thanks Riccardo. > > I am well aware of the submission form > > However, my question relates to doing submission within PyCharm itself. > > This is what I do at Pycharm *terminal* to invoke the module python > > spark-submit --jars > ..\lib\spark-bigquery-with-dependencies_2.12-0.18.0.jar \ > --packages com.github.samelamin:spark-bigquery_2.11:0.2.6 > analyze_house_prices_GCP.py > > However, at terminal run it does not pickup import dependencies in the > code! > > Traceback (most recent call last): > File > "C:/Users/admin/PycharmProjects/pythonProject2/DS/src/analyze_house_prices_GCP.py", > line 8, in <module> > import sparkstuff as s > ModuleNotFoundError: No module named 'sparkstuff' > > The python code is attached, pretty simple > > Thanks > > > >
