Try the following: df.map { case Row(id: Int, num: Int, value: Double, x: Float) => // replace those with your types (id, Vectors.dense(num, value, x)) }.toDF("id", "features")
-Xiangrui On Thu, Feb 26, 2015 at 3:08 PM, mobsniuk <mobsn...@gmail.com> wrote: > I've been searching around and see others have asked similar questions. > > Given a schemaRDD I extract a restless that contains numbers, both Int and > Doubles. How do I construct a RDD[Vector]? In 1.2 I wrote the results to a > textile and then read them back in splitting them with some code I found in > a ML book on Spark Analytics. That seems clunky. In 1.3 snapshot that flow > doesn't even work as I couldn't find a call to write the data out to a file. > > So I want to do two things with this data. Use the basic stats to extract > out some basic metrics from the data. The other as mentioned above is the ML > library. The ML Pipeline seems to use the new data frame API but the basic > stats still requires RDD[Vector]. > > Thanks > > Mark > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Converting-SchemaRDD-Dataframe-to-RDD-vector-tp21835.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org