https://spark.apache.org/docs/3.3.2/sql-ref-functions-udf-aggregate.html
I'm trying to run this example on Databricks, and it fails with the stacktrace below. It's literally a copy-paste from the example, what am I missing? Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: org.apache.spark.sql.TypedColumn Serialization stack: - object not serializable (class: org.apache.spark.sql.TypedColumn, value: myaverage(knownnotnull(assertnotnull(input[0, $line867a2458f6df413bb3920f7fc2dbb8c381.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$Average, true])).sum AS sum, knownnotnull(assertnotnull(input[0, $line867a2458f6df413bb3920f7fc2dbb8c381.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$Average, true])).count AS count, newInstance(class $line867a2458f6df413bb3920f7fc2dbb8c381.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$Average), boundreference()) AS average_salary)
