[
https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588934#comment-16588934
]
Erik Erlandson commented on SPARK-7768:
---------------------------------------
We use `UserDefinedType`, for example here:
[https://github.com/isarn/isarn-sketches-spark/blob/develop/src/main/scala/org/apache/spark/isarnproject/sketches/udt/TDigestUDT.scala#L37]
My colleague [~willbenton] and I gave a talk at Spark+AI summit in June on
[this topic|https://databricks.com/session/apache-spark-for-library-developers]
A comment about {{Encoder}}: they are strongly typed, which is quite nice to
work with in Scala but if you are intending to expose via DataFrame and/or
PySpark via py4j, they can't help you, and you need UDTs.
> Make user-defined type (UDT) API public
> ---------------------------------------
>
> Key: SPARK-7768
> URL: https://issues.apache.org/jira/browse/SPARK-7768
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Xiangrui Meng
> Priority: Critical
>
> As the demand for UDTs increases beyond sparse/dense vectors in MLlib, it
> would be nice to make the UDT API public in 1.5.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
