Re: Spark 2.3 Tree Error

2018-05-26 Thread Aakash Basu
I think I found the solution. The last comment from this link - https://issues.apache.org/jira/browse/SPARK-14948 But, my question is even after using table.column, why does Spark find the same column name from two different tables ambiguous? I mean table1.column = table2.column, Spark should co

Re: Spark 2.3 Tree Error

2018-05-26 Thread Aakash Basu
You're right. The same set of queries are working for max 2 columns in loop. If I give more than 2 column, the 2nd column is failing with this error - *attribute(s) with the same name appear in the operation: marginal_adhesion_bucketed. Please check if the right attribute(s) are used.* Any idea

Re: Spark 2.3 Tree Error

2018-05-26 Thread hemant singh
Per the sql plan this is where it is failing - Attribute(s) with the same name appear in the operation: fnlwgt_bucketed. Please check if the right attribute(s) are used.; On Sat, May 26, 2018 at 6:16 PM, Aakash Basu wrote: > Hi, > > This query is based on one step further from the query in th

Spark 2.3 Tree Error

2018-05-26 Thread Aakash Basu
Hi, This query is based on one step further from the query in this link . In this scenario, I add 1 or 2 more columns to be processed, Spark throws an ERROR by printing the physical plan of queries. It says,