It works. Is it better to have hive in this case for better performance ? On Thu, Jul 21, 2016 at 12:30 PM, Simone <simone.mirag...@gmail.com> wrote:
> If you have a folder, and a bunch of json inside that folder- yes it > should work. Just set as path something like "path/to/your/folder/*.json" > All files will be loaded into a dataframe and schema will be the union of > all the different schemas of your json files (only if you have different > schemas) > It should work - let me know > > Simone Miraglia > ------------------------------ > Da: Ashutosh Kumar <kmr.ashutos...@gmail.com> > Inviato: 21/07/2016 08:55 > A: Simone <simone.mirag...@gmail.com>; user @spark <user@spark.apache.org> > Oggetto: Re: Reading multiple json files form nested folders for data > frame > > That example points to a particular json file. Will it work same way if I > point to top level folder containing all json files ? > > On Thu, Jul 21, 2016 at 12:04 PM, Simone <simone.mirag...@gmail.com> > wrote: > >> Yes you can - have a look here >> http://spark.apache.org/docs/latest/sql-programming-guide.html#json-datasets >> >> Hope it helps >> >> Simone Miraglia >> ------------------------------ >> Da: Ashutosh Kumar <kmr.ashutos...@gmail.com> >> Inviato: 21/07/2016 08:19 >> A: user @spark <user@spark.apache.org> >> Oggetto: Reading multiple json files form nested folders for data frame >> >> I need to read bunch of json files kept in date wise folders and perform >> sql queries on them using data frame. Is it possible to do so? Please >> provide some pointers . >> >> Thanks >> Ashutosh >> > >
