Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23248#discussion_r239430315
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala
---
@@ -60,8 +60,12 @@ private class BatchIterator[T](iter: Iterator[T],
batchSize: Int)
/**
* A logical plan that evaluates a [[PythonUDF]].
*/
-case class ArrowEvalPython(udfs: Seq[PythonUDF], output: Seq[Attribute],
child: LogicalPlan)
- extends UnaryNode
+case class ArrowEvalPython(
+ udfs: Seq[PythonUDF],
+ output: Seq[Attribute],
+ child: LogicalPlan) extends UnaryNode {
+ override def producedAttributes: AttributeSet =
AttributeSet(output.drop(child.output.length))
--- End diff --
a different but related fix, to make the `missingAttributes` calculated
correctly.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
