For Spark 1.6.x, a DataFrame can't be directly converted to a Dataset[Row], but can done indirectly as follows:
import org.apache.spark.sql.catalyst.encoders.RowEncoder // assume df is a DataFrame implicit val encoder: ExpressionEncoder[Row] = RowEncoder(df.schema) val ds = df.as[Row] However, it may be more convenient to convert a DataFrame to a Dataset of Tuple or case class corresponding to the row schema. > On Jul 16, 2016, at 03:21, Daniel Barclay <danielbarclay....@gmail.com> wrote: > > In Spark 1.6.1, how can I convert a DataFrame to a Dataset[Row]? > > Is there a direct conversion? (Trying <someDataframe>.as[Row] doesn't work, > even after importing <my sqlContext>.implicits._ .) > > Is there some way to map the Rows from the Dataframe into the Dataset[Row]? > (DataFrame.map would just make another Dataframe, right?) > > > Thanks, > Daniel > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >
