Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21318#discussion_r187857692
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -39,7 +39,21 @@ import org.apache.spark.util.Utils
/**
- * Functions available for DataFrame operations.
+ * Commonly used functions available for DataFrame operations. Using
functions defined here provides
+ * a little bit more compile-time safety to make sure the function exists.
+ *
+ * Spark also includes more built-in functions that are less common and
are not defined here.
+ * You can still access them (and all the functions defined here) using
the [[functions.expr()]] API
+ * and calling them through a SQL expression string. You can find the
entire list of functions for
+ * the latest version of Spark at
[[https://spark.apache.org/docs/latest/api/sql/index.html]].
+ *
+ * As an example, `isnan` is a function that is defined here. You can use
`isnan(col("myCol"))`
+ * to invoke the isnan function. This way the programming language's
compiler ensures isnan exists
+ * and is of the proper form. You can also use `expr("isnan(myCol)")`
function to invoke the same
+ * function. In this case, Spark itself will ensure isnan exists when it
analyzes the query.
--- End diff --
nit: `isnan`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
