Re: Any equivalent method lateral and explore

Hi, Interesting, but I personally would opt for withColumn since it'd be less to type (and also be consistent with ticks (')) as follows: df.withColumn(explode('myArray) as 'arrayItem) (Spark SQL made my SQL developer's life so easy these days :)) Pozdrawiam, Jacek Laskowski

Re: Any equivalent method lateral and explore

Both collect_list and explode are available in the function library . The following is an example of using it: df.select($"*", explode($"myArray") as 'arrayItem) On Tue, Nov 22, 2016 at 2:42 PM, Mahender Sarangam

Any equivalent method lateral and explore

Hi, We are converting our hive logic which is using lateral view and explode functions. Is there any builtin function in scala for performing lateral view explore. Below is our query in Hive. temparray is temp table with c0 and c1 columns SELECT id, CONCAT_WS(',', collect_list(LineID)) as