Re: write parquet with statistics min max with binary field

2018-01-28 Thread Stephen Joung
After setting `parquet.strings.signed-min-max.enabled` to `true` in `ShowMetaCommand.java`, parquet-tools meta show min,max. @@ -57,8 +57,9 @@ public class ShowMetaCommand extends ArgsOnlyCommand { String[] args = options.getArgs(); String input = args[0];

Re: write parquet with statistics min max with binary field

2018-01-23 Thread Stephen Joung
How can I write parquet file with min/max statistic? 2018-01-24 10:30 GMT+09:00 Stephen Joung : > Hi, I am trying to use spark sql filter push down. and specially want to > use row group skipping with parquet file. > > And I guessed that I need parquet file with statistics

write parquet with statistics min max with binary field

2018-01-23 Thread Stephen Joung
Hi, I am trying to use spark sql filter push down. and specially want to use row group skipping with parquet file. And I guessed that I need parquet file with statistics min/max. On spark master branch - I tried to write single column with "a", "b", "c" to parquet file f1 scala>