Re: [Error] : dynamically union All + adding new column

2016-03-19 Thread Ted Yu
bq. .drop("Col9") Could it be due to the above ? On Wed, Mar 16, 2016 at 7:29 PM, Divya Gehlot wrote: > Hi, > I am dynamically doing union all and adding new column too > > val dfresult = >>

Re: [Error] : dynamically union All + adding new column

2016-03-19 Thread Ted Yu
It turned out that Col1 appeared twice in the select :-) > On Mar 16, 2016, at 7:29 PM, Divya Gehlot wrote: > > Hi, > I am dynamically doing union all and adding new column too > >> val dfresult = >>

[Error] : dynamically union All + adding new column

2016-03-19 Thread Divya Gehlot
Hi, I am dynamically doing union all and adding new column too val dfresult = > dfAcStamp.select("Col1","Col1","Col3","Col4","Col5","Col6","col7","col8","col9") > val schemaL = dfresult.schema > var dffiltered = sqlContext.createDataFrame(sc.emptyRDD[Row], schemaL) > for ((key,values) <- lcrMap)