Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22610 > he only other idea I have is to provide an option to raise an error if the type needs to be cast. Actually sounds good to me. I think the problem is we are not quite clear about when the type is mismatched in UDFs (see also https://github.com/apache/spark/pull/20163 for a reminder). IIRC, we rather roughly agreed upon documenting it (and allowing exact type match (?)). @viirya and @BryanCutler, how about we document that return types should be matched (we can leave a chart or map referring (`types.to_arrow_type`)? One additional improvement might be .. we describe that type casting behaviour is .. say .. not guaranteed but I am not sure how we can nicely document this. Probably only mentioning the type mapping is fine ..?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org