Github user e-dorigatti commented on the issue: https://github.com/apache/spark/pull/21383 Yes, the problem was that the signature is lost when the function is wrapped, and the worker needs the signature to know whether the function needs keys together with values or not. What I meant is that fixing worker side might fix this specific JIRA, but the bug will still occur when an udf is not used in a worker, but somewhere else. If you fix at the udf side, however, the bug will never occur again, regardless of how and where the udf is used. I don't know the codebase well enough to know whether this will be a real problem, though
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org