read all parquet files in a directory in spark-sql

2014-10-13 Thread Sadhan Sood
How can we read all parquet files in a directory in spark-sql. We are following this example which shows a way to read one file: // Read in the parquet file created above. Parquet files are self-describing so the schema is preserved.// The result of loading a Parquet file is also a SchemaRDD.val

Re: read all parquet files in a directory in spark-sql

2014-10-13 Thread Nicholas Chammas
...@gmail.com wrote: How can we read all parquet files in a directory in spark-sql. We are following this example which shows a way to read one file: // Read in the parquet file created above. Parquet files are self-describing so the schema is preserved.// The result of loading a Parquet file

Re: read all parquet files in a directory in spark-sql

2014-10-13 Thread DB Tsai
we read all parquet files in a directory in spark-sql. We are following this example which shows a way to read one file: // Read in the parquet file created above. Parquet files are self-describing so the schema is preserved. // The result of loading a Parquet file is also a SchemaRDD. val

Re: read all parquet files in a directory in spark-sql

2014-10-13 Thread Sadhan Sood
on Parquet files to request this feature. Nick On Mon, Oct 13, 2014 at 12:21 PM, Sadhan Sood sadhan.s...@gmail.com wrote: How can we read all parquet files in a directory in spark-sql. We are following this example which shows a way to read one file: // Read in the parquet file created