Re: How to read compressed parquet file

[Cheng Lian]

You need to use "har://" instead of "hdfs://" to read HAR files. Just 
tested against Spark 1.5, and it works as expected.


Cheng

On 9/9/15 3:29 PM, 李铖 wrote:
I think too many parquet files may be affect reading capability,so I 
use hadoop archive to combine them,but 
 sql_context.read.parquet(output_path) does not work on the file.

How to fix it ,please help me.
：）

How to read compressed parquet file

[李铖]

I think too many parquet files may be affect reading capability,so I use
hadoop archive to combine them,but  sql_context.read.parquet(output_path)
does not work on the file.
How to fix it ,please help me.
：）

Re: How to read compressed parquet file

[李铖]

It works. at spark 1.4
Thanks a lot.

2015-09-09 17:21 GMT+08:00 Cheng Lian :

> You need to use "har://" instead of "hdfs://" to read HAR files. Just
> tested against Spark 1.5, and it works as expected.
>
> Cheng
>
>
> On 9/9/15 3:29 PM, 李铖 wrote:
>
> I think too many parquet files may be affect reading capability,so I use
> hadoop archive to combine them,but  sql_context.read.parquet(output_path)
> does not work on the file.
> How to fix it ,please help me.
> ：）
>
>
>