[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[
https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033376#comment-17033376
]
Liang Zhang commented on SPARK-30762:
-
Hi Hyukjin, I updated the description.
> Add dtype="float32" support to vector_to_array UDF
> --
>
> Key: SPARK-30762
> URL: https://issues.apache.org/jira/browse/SPARK-30762
> Project: Spark
> Issue Type: Story
> Components: MLlib, PySpark
>Affects Versions: 3.0.0
>Reporter: Liang Zhang
>Assignee: Liang Zhang
>Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Previous PR:
> [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py]
> In the previous PR, we introduced a UDF to convert a column of MLlib Vecters
> to a column of lists in python (Seq in scala). Currently, all the floating
> numbers in a vector is converted to Double in scala. In this issue, we will
> add a parameter in the python function {{vector_to_array(col)}} that allows
> converting to Float (32bits) in scala, which would be mapped to a numpy array
> of dtype=float32.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033346#comment-17033346 ] Hyukjin Kwon commented on SPARK-30762: -- Sorry, can you clarify what you mean by dtype support? > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033151#comment-17033151 ] Liang Zhang commented on SPARK-30762: - I'm now working on this issue. > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
