MaxGekk commented on code in PR #57184:
URL: https://github.com/apache/spark/pull/57184#discussion_r3577103080
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -9453,6 +13624,8 @@ object functions {
* default column name `pos` for position, and `col` for elements in the
array and `key` and
* `value` for elements in the map unless specified otherwise.
*
+ * @param e
+ * the target column to explode. A column that evaluates to an array or a
map.
Review Comment:
`posexplode` (and `posexplode_outer` just below) got a `@param` but no
`@return`, whereas the parallel `explode`/`explode_outer` document one ("a
column of the element type of the input array, or the key and value columns of
the input map"). Worth mirroring `explode`'s `@return` here, plus the added
`pos` position column, so the generator family is documented consistently.
(Non-blocking.)
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -9548,6 +13745,8 @@ object functions {
*
* @group json_funcs
* @since 2.2.0
+ * @return
+ * Returns a column that evaluates to a struct.
Review Comment:
This `@return` ("evaluates to a struct") is accurate for the `StructType`
overloads, but the overloads taking `schema: DataType`, `schema: String`, or
`schema: Column` also accept `ArrayType` and `MapType` top-level schemas — the
description just above even says "a `MapType` ... `StructType` or `ArrayType`
with the specified schema", and `JsonToStructs.dataType` is the schema itself.
Verified against a local build:
```
from_json(col, ArrayType(IntegerType)) ->
ArrayType(IntegerType,true)
from_json(col, MapType(StringType, IntegerType)) ->
MapType(StringType,IntegerType,true)
from_json(col, lit("array<int>")) ->
ArrayType(IntegerType,true)
from_json(col, StructType(...)) -> StructType(...) //
the only struct case
```
So `from_json(json, "array<int>")` returns an array, not a struct. Consider,
on the non-`StructType` overloads, something like:
```suggestion
* Returns a column of the type given by the schema (a struct, array, or
map).
```
(Non-blocking. The `StructType` overloads' `@return` is correct as-is.)
##########
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -5904,24 +8598,34 @@ object functions {
* Only allowed at the end of the format string; specifies that 'expr'
indicates a negative
* number with wrapping angled brackets.
*
+ * @param e
+ * The input string to convert to a number. A column that evaluates to a
string.
+ * @param format
+ * The format to use to convert the value. A column that evaluates to a
string. Must be a
+ * constant.
* @group string_funcs
* @since 3.5.0
+ * @return
+ * Returns a column that evaluates to a decimal.
*/
def to_number(e: Column, format: Column): Column = Column.fn("to_number", e,
format)
/**
* Replaces all occurrences of `search` with `replace`.
*
* @param src
- * A column of string to be replaced
+ * A column of string to be replaced. A column that evaluates to a string.
* @param search
- * A column of string, If `search` is not found in `str`, `str` is
returned unchanged.
+ * A column of string, If `search` is not found in `str`, `str` is
returned unchanged. A
Review Comment:
Minor: "A column of string, If `search` is not found ..." has a comma splice
(the comma before "If" should be a period) and "of string" should be "of
strings". This text is pre-existing, but since this PR edits the line (appends
the type sentence), it's cheap to tidy in passing. (Nit.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
