Spark SQL: what does an exclamation mark mean in the plan?

2015-10-19 Thread Xiao Li
Hi, all, After turning on the trace, I saw a strange exclamation mark in the intermediate plans. This happened in catalyst analyzer. Join Inner, Some((col1#0 = col1#6)) Project [col1#0,col2#1,col3#2,col2_alias#24,col3#2 AS col3_alias#13] Project [col1#0,col2#1,col3#2,col2#1 AS col2_alias#24]

Re: Spark SQL: what does an exclamation mark mean in the plan?

2015-10-19 Thread Michael Armbrust
It means that there is an invalid attribute reference (i.e. a #n where the attribute is missing from the child operator). On Sun, Oct 18, 2015 at 11:38 PM, Xiao Li wrote: > Hi, all, > > After turning on the trace, I saw a strange exclamation mark in > the intermediate

Re: Spark SQL: what does an exclamation mark mean in the plan?

2015-10-19 Thread Xiao Li
Hi, Michael, Thank you again! Just found the functions that generate the ! mark /** * A prefix string used when printing the plan. * * We use "!" to indicate an invalid plan, and "'" to indicate an unresolved plan. */ protected def statePrefix = if (missingInput.nonEmpty &&