Hi,
We implemented an External Data Source by extending the TableScan . We added
the classes to the classpath
The data source works fine when run in Spark Shell .
But currently we are unable to use this same data source in Python Environment.
So when we execute the following below in an Ipython notebook
sqlContext.sql("CREATE TEMPORARY TABLE dataTable USING MyDataSource OPTIONS
(partitions '2')") we get the following error :
Py4JJavaError: An error occurred while calling o78.sql.
: java.lang.RuntimeException: Failed to load class for data source: MyDataSource
How to expose this data source for consumption even in PySpark environment.
Regards,
Santosh
