I submitted a PR for this:
https://github.com/apache/spark/pull/25024
On Wed, Mar 27, 2019 at 4:19 PM Erik Erlandson wrote:
> I describe some of the details here:
> https://issues.apache.org/jira/browse/SPARK-27296
>
> The short version of the story is that aggregating data structures (UDTs)
>
Hi all,
In the past two years, the pandas UDFs are perhaps the most important changes
to Spark for Python data science. However, these functionalities have evolved
organically, leading to some inconsistencies and confusions among users. I
created a ticket and a document summarizing the issues,
