No, you would just have to do another select to pull out the fields you are interested in.
On Sat, Sep 26, 2015 at 11:11 AM, Jerry Lam <chiling...@gmail.com> wrote: > Hi Michael, > > Thanks for the tip. With dataframe, is it possible to explode some > selected fields in each purchase_items? > Since purchase_items is an array of item and each item has a number of > fields (for example product_id and price), is it possible to just explode > these two fields directly using dataframe? > > Best Regards, > > > Jerry > > On Fri, Sep 25, 2015 at 7:53 PM, Michael Armbrust <mich...@databricks.com> > wrote: > >> The SQL parser without HiveContext is really simple, which is why I >> generally recommend users use HiveContext. However, you can do it with >> dataframes: >> >> import org.apache.spark.sql.functions._ >> table("purchases").select(explode(df("purchase_items")).as("item")) >> >> >> >> On Fri, Sep 25, 2015 at 4:21 PM, Jerry Lam <chiling...@gmail.com> wrote: >> >>> Hi sparkers, >>> >>> Anyone knows how to do LATERAL VIEW EXPLODE without HiveContext? >>> I don't want to start up a metastore and derby just because I need >>> LATERAL VIEW EXPLODE. >>> >>> I have been trying but I always get the exception like this: >>> >>> Name: java.lang.RuntimeException >>> Message: [1.68] failure: ``union'' expected but identifier view found >>> >>> with the query look like: >>> >>> "select items from purhcases lateral view explode(purchase_items) tbl as >>> items" >>> >>> Best Regards, >>> >>> Jerry >>> >>> >> >