[GitHub] spark pull request #21820: [SPARK-24868][PYTHON]add sequence function in Pyt...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/21820


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21820: [SPARK-24868][PYTHON]add sequence function in Pyt...

[huaxingao]

Github user huaxingao commented on a diff in the pull request:

https://github.com/apache/spark/pull/21820#discussion_r203934505
  
--- Diff: python/pyspark/sql/functions.py ---
@@ -2551,6 +2551,27 @@ def map_concat(*cols):
 return Column(jc)
 
 
+@since(2.4)
+def sequence(start, stop, step=None):
+"""
+Generate a sequence of integers from start to stop, incrementing by 
step.
+If step is not set, incrementing by 1 if start is less than or equal 
to stop, otherwise -1.
+
+>>> df1 = spark.createDataFrame([(-2, 2)], ('C1', 'C2'))
+>>> df1.select(sequence('C1', 'C2').alias('r')).collect()
+[Row(r=[-2, -1, 0, 1, 2])]
+>>> df2 = spark.createDataFrame([(4, -4, -2)], ('C1', 'C2', 'C3'))
--- End diff --

It will throw ```java.lang.IllegalArgumentException```. 
There is a check in ```collectionOperations.scala```
```
require(
  (step > num.zero && start <= stop)
|| (step < num.zero && start >= stop)
|| (step == num.zero && start == stop),
  s"Illegal sequence boundaries: $start to $stop by $step")
```


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21820: [SPARK-24868][PYTHON]add sequence function in Pyt...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/21820#discussion_r203924999
  
--- Diff: python/pyspark/sql/functions.py ---
@@ -2551,6 +2551,27 @@ def map_concat(*cols):
 return Column(jc)
 
 
+@since(2.4)
+def sequence(start, stop, step=None):
+"""
+Generate a sequence of integers from start to stop, incrementing by 
step.
+If step is not set, incrementing by 1 if start is less than or equal 
to stop, otherwise -1.
+
+>>> df1 = spark.createDataFrame([(-2, 2)], ('C1', 'C2'))
+>>> df1.select(sequence('C1', 'C2').alias('r')).collect()
+[Row(r=[-2, -1, 0, 1, 2])]
+>>> df2 = spark.createDataFrame([(4, -4, -2)], ('C1', 'C2', 'C3'))
--- End diff --

What happens if the `step` is positive in this case?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org