Hi all, I found that for the same parquet file, using pq.ParquetFile(file_name).read() takes 6s while pq.read_table(file_name) takes 17s. How do those two apis differ? I thought they use the same internals but it seems not. The parquet file is 865MB, snappy compression and enable dictionary. All other settings are default, writing with pyarrow.
- [Python][Parquet]pq.ParquetFile.read faster than pq.read_tabl... Shawn Zeng
- Re: [Python][Parquet]pq.ParquetFile.read faster than pq.... Weston Pace
- Re: [Python][Parquet]pq.ParquetFile.read faster than... Shawn Zeng
- Re: [Python][Parquet]pq.ParquetFile.read faster ... Weston Pace
- Re: [Python][Parquet]pq.ParquetFile.read fas... Shawn Zeng
- Re: [Python][Parquet]pq.ParquetFile.rea... Weston Pace
- Re: [Python][Parquet]pq.ParquetFile... Wes McKinney
- Re: [Python][Parquet]pq.Parquet... Weston Pace
