Hi all,
I have two questions about selecting columns of Dataset.
First, could you tell me know if there is any way to select TypedColumn
columns in addition to the combination of `expr` and `as`?
Second, how can we alias such a `expr("name").as[String]` Column?
I tried to select a column of Dataset like DataFrame.
However, I couldn't do that.
```
case class Person(id: Int, name: String)
val df = sc.parallelize(Seq((1, "Bob"), (2, "Tom"))).toDF("id", "name")
val ds = df.as[Person]
ds.select(expr("name").as[String]).show
+-----+
|value|
+-----+
| Bob|
| Tom|
+-----+
ds.select('id).show
<console>:34: error: type mismatch;
found : Symbol
required: org.apache.spark.sql.TypedColumn[Person,?]
ds.select('id).show')
ds.select($"id").show
<console>:34: error: type mismatch;
found : org.apache.spark.sql.ColumnName
required: org.apache.spark.sql.TypedColumn[Person,?]
ds.select($"id").show
ds.select(ds("id")).show
<console>:34: error: org.apache.spark.sql.Dataset[Person] does not take
parameters
ds.select(ds("id")).show
ds.select("id").show
<console>:34: error: type mismatch;
found : String("id")
required: org.apache.spark.sql.TypedColumn[Person,?]
ds.select("id").show
```
Best,
Yu
-----
-- Yu Ishikawa
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Is-there-any-way-to-select-columns-of-Dataset-in-addition-to-the-combination-of-expr-and-as-tp15713.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
