Hi,
We are using HDP3.0 (using zeppelin 0.8.0) and are migrating Jupyter
notebooks to Zeppelin. One issue we came across is that a python script
calling R script does not work in Zeppelin.
%livy2.pyspark
import os
sc.addPyFile("hdfs:///user/zeppelin/my.py")
import my
my.test()
my.test() calls R script like: ['Rscript', 'myR.r']
Fatal error: cannot open file 'myR.r': No such file or directory
When running this notebook in jupyter, both my.py and myR.r exist in the
same folder. I understand the story changes on hadoop because the scripts
run in containers.
My question:
Is this scenario supported in zeppelin? How to add a R script into a python
spark context so that the Python script can find the R script? Appreciate!
