Re: Expected benefit of parquet filter pushdown?

2016-09-01 Thread Christon DeWan
> - Robert > > On 8/31/16, 10:29 PM, "cde...@apple.com on behalf of Christon DeWan" > <cde...@apple.com> wrote: > >I have a data set stored in parquet with several short key fields and one > relatively large (several kb) blob field. The data set

Expected benefit of parquet filter pushdown?

2016-08-31 Thread Christon DeWan
I have a data set stored in parquet with several short key fields and one relatively large (several kb) blob field. The data set is sorted by key1, key2. message spark_schema { optional binary key1 (UTF8); optional binary key2; optional binary blob; } One use case of