Re: parsing embedded json in spark

2016-12-23 Thread Tal Grynbaum
Hi Shaw, Thanks, that works! On Thu, Dec 22, 2016 at 6:45 PM, Shaw Liu wrote: > Hi,I guess you can use 'get_json_object' function > > Get Outlook for iOS > > > > > On Thu, Dec 22, 2016 at 9:52 PM +0800, "Irving Duran" < > irving.du...@gmail.com> wrote: > > Is it an opt

Re: parsing embedded json in spark

2016-12-22 Thread Shaw Liu
Hi,I guess you can use 'get_json_object' function Get Outlook for iOS On Thu, Dec 22, 2016 at 9:52 PM +0800, "Irving Duran" wrote: Is it an option to parse that field prior of creating the dataframe? If so, that's what I would do. In terms of your master node only working, you hav

Re: parsing embedded json in spark

2016-12-22 Thread Irving Duran
Is it an option to parse that field prior of creating the dataframe? If so, that's what I would do. In terms of your master node only working, you have to share more about your structure, are you using spark standalone, yarn, or mesos? Thank You, Irving Duran On Thu, Dec 22, 2016 at 1:42 AM, T