Sean Zhong created SPARK-16898: ---------------------------------- Summary: Adds argument type information for typed logical plan likMapElements, TypedFilter, and AppendColumn Key: SPARK-16898 URL: https://issues.apache.org/jira/browse/SPARK-16898 Project: Spark Issue Type: Bug Reporter: Sean Zhong Priority: Minor
Typed logical plan like MapElements, TypedFilter, and AppendColumn contains a closure field: {{func: (T) => Boolean}}. For example class TypedFilter's signature is: {code} case class TypedFilter( func: AnyRef, deserializer: Expression, child: LogicalPlan) extends UnaryNode {code} >From the above class signature, we cannot easily find: 1. What is the input argument's type of the closure {{func}}? How do we know which apply method to pick if there are multiple overloaded apply methods? 2. What is the input argument's schema? With this info, it is easier for us to define some custom optimizer rule to translate these typed logical plan to more efficient implementation, like the closure optimization idea in SPARK-14083. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org