Hi, Interesting, but I personally would opt for withColumn since it'd be less to type (and also be consistent with ticks (')) as follows:
df.withColumn(explode('myArray) as 'arrayItem) (Spark SQL made my SQL developer's life so easy these days :)) Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Tue, Nov 22, 2016 at 11:58 PM, Michael Armbrust <mich...@databricks.com> wrote: > Both collect_list and explode are available in the function library. > > The following is an example of using it: > df.select($"*", explode($"myArray") as 'arrayItem) > > On Tue, Nov 22, 2016 at 2:42 PM, Mahender Sarangam > <mahender.bigd...@outlook.com> wrote: >> >> Hi, >> >> We are converting our hive logic which is using lateral view and explode >> functions. Is there any builtin function in scala for performing lateral >> view explore. >> >> >> Below is our query in Hive. temparray is temp table with c0 and c1 >> columns >> >> SELECT id, CONCAT_WS(',', collect_list(LineID)) as LineiD >> FROM (SELECT cast(LineID as STRING) as LineiD, cast(id as STRING) as id >> FROM temparray LATERAL VIEW explode(`_c1`) adTable AS id) T >> GROUP BY id; >> >> >> Can any one provide pointer for string functions available in scala. We >> would like perform operations like Collect_List, get starting index of >> matching string. >> >> >> >> Nanu >> >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org