Re: Using Dataframe API vs. RDD API?

2018-01-04 Thread Pat Ferrel
Actually there are libs that will read DFs from HBase https://svn.apache.org/repos/asf/hbase/hbase.apache.org/trunk/_chapters/spark.html This is out of band with PIO and should not be used IMO because the

Re: Using Dataframe API vs. RDD API?

2018-01-04 Thread Pat Ferrel
Funny you should ask this. Yes, we are working on a DF based Universal Recommender but you have to convert the RDD into a DF since PIO does not read out data in the form of a DF (yet). This is a fairly simple step of maybe one line of code but would be better supported in PIO itself. The issue