Repository: spark Updated Branches: refs/heads/branch-2.2 1e4cca02f -> 1cf3e3a26
[SPARK-22862] Docs on lazy elimination of columns missing from an encoder This behavior has confused some users, so lets clarify it. Author: Michael Armbrust <mich...@databricks.com> Closes #20048 from marmbrus/datasetAsDocs. (cherry picked from commit 8df1da396f64bb7fe76d73cd01498fdf3b8ed964) Signed-off-by: gatorsmile <gatorsm...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1cf3e3a2 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1cf3e3a2 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1cf3e3a2 Branch: refs/heads/branch-2.2 Commit: 1cf3e3a26961d306eb17b7629d8742a4df45f339 Parents: 1e4cca0 Author: Michael Armbrust <mich...@databricks.com> Authored: Thu Dec 21 21:38:16 2017 -0800 Committer: gatorsmile <gatorsm...@gmail.com> Committed: Thu Dec 21 21:38:34 2017 -0800 ---------------------------------------------------------------------- sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala | 4 ++++ 1 file changed, 4 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/1cf3e3a2/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---------------------------------------------------------------------- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala b/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala index 1acbad9..fb700a4 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala @@ -368,6 +368,10 @@ class Dataset[T] private[sql]( * If the schema of the Dataset does not match the desired `U` type, you can use `select` * along with `alias` or `as` to rearrange or rename as required. * + * Note that `as[]` only changes the view of the data that is passed into typed operations, + * such as `map()`, and does not eagerly project away any columns that are not present in + * the specified class. + * * @group basic * @since 1.6.0 */ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
