dffiltered = unionAll(dfresult.filter(dfFilterSQLs(
i)).select("Col1","Col2","Col3","Col4","Col5"))
FYI
On Sun, Mar 13, 2016 at 8:50 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> Have you tried unionAll() method of DataFrame ?
>
> On Sun, Mar 13, 2016 at 8:44 PM, Divya Gehlot <divya.htco...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Please bear me for asking such a naive question
>> I have list of conditions (dynamic sqls) sitting in hbase table .
>> I need to iterate through those dynamic sqls and add the data to
>> dataframes.
>> As we know dataframes are immutable ,when I try to iterate in for loop as
>> shown below I get only last dynamic sql result set .
>>
>> var dffiltered : DataFrame = sqlContext.emptyDataFrame
>> for ( i <- 0 to (dfFilterSQLs.length - 1)) {
>> println("Condition============="+dfFilterSQLs(i))
>> dffiltered =
>> dfresult.filter(dfFilterSQLs(i)).select("Col1","Col2","Col3","Col4","Col5")
>> dffiltered.show
>> }
>>
>>
>> How can I keep on appending data to dataframe and get the final result
>> having all the sql conditions.
>>
>> Thanks in advance for the help.
>>
>> Thanks,
>> Divya
>>
>
>
