zhengruifeng commented on code in PR #46519: URL: https://github.com/apache/spark/pull/46519#discussion_r1596217131
########## python/pyspark/sql/connect/group.py: ########## @@ -34,6 +34,7 @@ from pyspark.util import PythonEvalType from pyspark.sql.group import GroupedData as PySparkGroupedData from pyspark.sql.pandas.group_ops import PandasCogroupedOps as PySparkPandasCogroupedOps +from pyspark.sql.pandas.functions import _validate_pandas_udf # type: ignore[attr-defined] Review Comment: Spark Classic invoke `pandas_udf` in Pandas Functions (ApplyInXXX), `pandas_udf` includes the function validation. While in Spark Connect, we can not use `pandas_udf` due to the differences in underlying implementations: `pandas_udf` returns a wrapper while Spark Connect requires a `UserDefinedFunction` object. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
