Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20567 Regarding the error message, this is a good example to show how to provide a user-friendly message. To the external end users, they might not be aware of the internal implementation. They might not be aware of Apache ARROW is being used. They might not even know what is Apache Arrow. The conf might be set by the system admin or others. Thus, this error message is confusing to them. ``` ... py4j.protocol.Py4JJavaError: An error occurred while calling o42.collectAsArrowToPython. ... java.lang.UnsupportedOperationException: Unsupported data type: map<string,bigint> ``` Ideally, we could let users know how to bypass the issue. For example, let them disable the conf `spark.sql.execution.arrow.enabled`.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org