On 04/10/2015 04:24 PM, Tianqi Tong wrote:
Hi Parquet,
Is there anywhere that I can find the documentation about the explanation and
relationships for the following configurations:
set PARQUET_FILE_SIZE=x;
set parquet.block.size=y;
set dfs.blocksize=z;
Right now I'm populating a table but hard to find the best configuration of
those parameters.
Thanks!
Tianqi Tong
Tianqi,
Here's a post I wrote on row group and block sizes:
http://ingest.tips/2015/01/31/parquet-row-group-size/
I'm not sure what PARQUET_FILE_SIZE is. What are you using to write?
rb
--
Ryan Blue
Software Engineer
Cloudera, Inc.
