Is it possible to concisely create a dataset from a dataframe with fewer
columns? Specifically, suppose I create a dataframe with:
val df: DataFrame = Seq(("v1"),("v2")).toDF("f1")
Then, I have a case class for a dataset defined as:
case class CC(f1: String, f2: Option[String] = None)
I’d like to use df.as[CC] to get an instance of the case class, but this gives
me the following error:
org.apache.spark.sql.AnalysisException: cannot resolve '`f2`' given input
columns: [f1];
Is there a concise way to use the default values as defined by the case class
so that I don't have to explicitly add those columns to the dataframe before
converting to a dataset?
Warmest Regards,
Jason Tokayer, PhD
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
