[
https://issues.apache.org/jira/browse/SPARK-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188319#comment-15188319
]
Joseph K. Bradley commented on SPARK-13068:
-------------------------------------------
You're right that the current implementation would not support nested types
well. But I don't think we need full-blown ParamValidators; we really need a
separate concept in Python than in Scala: Python needs conversion, whereas
Scala can handle validation.
What if, instead of expectedType, we used a new field "typeConverter." It
could be given as an argument to Param and used where expectedType currently is
used.
We could deprecate expectedType for 2.0 and remove it in 2.1.
How does that sound?
> Extend pyspark ml paramtype conversion to support lists
> -------------------------------------------------------
>
> Key: SPARK-13068
> URL: https://issues.apache.org/jira/browse/SPARK-13068
> Project: Spark
> Issue Type: Improvement
> Components: ML, PySpark
> Reporter: holdenk
> Priority: Trivial
>
> In SPARK-7675 we added type conversion for PySpark ML params. We should
> follow up and support param type conversion for lists and nested structures
> as required. This blocks having all PySpark ML params having type information.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
