Re: [Spark SQL]: Convert JavaSchemaRDD back to JavaRDD of a specific class

[Cheng Lian]

Currently there’s no convenient way to convert a 
|SchemaRDD|/|JavaSchemaRDD| back to an |RDD|/|JavaRDD| of some case 
class. But you can convert an |RDD|/|JavaRDD| into an 
|RDD[Row]|/|JavaRDDRow| using |schemaRdd.rdd| and |new 
JavaRDDRow(schemaRdd.rdd)|.


Cheng

On 3/15/15 10:22 PM, Renato Marroquín Mogrovejo wrote:


Hi Spark experts,

Is there a way to convert a JavaSchemaRDD (for instance loaded from a 
parquet file) back to a JavaRDD of a given case class? I read on 
stackOverFlow[1] that I could do a select over the parquet file and 
then by reflection get the fields out, but I guess that would be an 
overkill.
Then I saw [2] from 2014 which says that this feature would be 
available in the future. So could you please let me know how I can 
accomplish this? Thanks in advance!



Renato M.

[1] 
http://stackoverflow.com/questions/26181353/how-to-convert-spark-schemardd-into-rdd-of-my-case-class
[2] 
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Convert-SchemaRDD-back-to-RDD-td9071.html


​

[Spark SQL]: Convert JavaSchemaRDD back to JavaRDD of a specific class

[Renato Marroquín Mogrovejo]

Hi Spark experts,

Is there a way to convert a JavaSchemaRDD (for instance loaded from a
parquet file) back to a JavaRDD of a given case class? I read on
stackOverFlow[1] that I could do a select over the parquet file and then by
reflection get the fields out, but I guess that would be an overkill.
Then I saw [2] from 2014 which says that this feature would be available in
the future. So could you please let me know how I can accomplish this?
Thanks in advance!


Renato M.

[1]
http://stackoverflow.com/questions/26181353/how-to-convert-spark-schemardd-into-rdd-of-my-case-class
[2]
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Convert-SchemaRDD-back-to-RDD-td9071.html