This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ddf4a50 [SPARK-28031][PYSPARK][TEST] Improve doctest on over function
of Column
ddf4a50 is described below
commit ddf4a5031287c0c26ea462dd89ea99d769473213
Author: Liang-Chi Hsieh <vii...@gmail.com>
AuthorDate: Thu Jun 13 11:04:41 2019 +0900
[SPARK-28031][PYSPARK][TEST] Improve doctest on over function of Column
## What changes were proposed in this pull request?
Just found the doctest on `over` function of `Column` is commented out. The
window spec is also not for the window function used there.
We should either remove the doctest, or improve it.
Because other functions of `Column` have doctest generally, so this PR
tries to improve it.
## How was this patch tested?
Added doctest.
Closes #24854 from viirya/column-test-minor.
Authored-by: Liang-Chi Hsieh <vii...@gmail.com>
Signed-off-by: HyukjinKwon <gurwls...@apache.org>
---
python/pyspark/sql/column.py | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/sql/column.py b/python/pyspark/sql/column.py
index e7dec11..7f12d23 100644
--- a/python/pyspark/sql/column.py
+++ b/python/pyspark/sql/column.py
@@ -668,9 +668,17 @@ class Column(object):
:return: a Column
>>> from pyspark.sql import Window
- >>> window = Window.partitionBy("name").orderBy("age").rowsBetween(-1,
1)
+ >>> window = Window.partitionBy("name").orderBy("age") \
+ .rowsBetween(Window.unboundedPreceding, Window.currentRow)
>>> from pyspark.sql.functions import rank, min
- >>> # df.select(rank().over(window), min('age').over(window))
+ >>> df.withColumn("rank", rank().over(window)) \
+ .withColumn("min", min('age').over(window)).show()
+ +---+-----+----+---+
+ |age| name|rank|min|
+ +---+-----+----+---+
+ | 5| Bob| 1| 5|
+ | 2|Alice| 1| 2|
+ +---+-----+----+---+
"""
from pyspark.sql.window import WindowSpec
if not isinstance(window, WindowSpec):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
