A incomplete map to a maze - or - the jurney down the rabbit hole

Hi, I have been meaning to write a blog post regarding our Drill experiments but I though I might share some thoughts here first. Hopefully some newbie can benefit from this and perhaps it sheds some light on what drives newcomers in this community (or at least some part of them). A bit of a back

Re: A incomplete map to a maze - or - the jurney down the rabbit hole

For the issues about partition pruning over directories, could you please provide more detail information? Drill should do partition pruning based on directory. If it does not work the way you want, probably there is a bug in the code. We would appreciate if you can provide more detail, so that w

Re: A incomplete map to a maze - or - the jurney down the rabbit hole

As Jinfeng mentioned, directory based partition pruning should work. You might also be interested in DRILL- which allows you to auto partition data when using CTAS. - Rahul On Mon, Aug 3, 2015 at 5:03 PM, Jinfeng Ni wrote: > For the issues

Re: A incomplete map to a maze - or - the jurney down the rabbit hole

Hi, You are right, partition pruning works using simple queries (I will adjust our queries accordingly): - select count(*) from dfs.tmp.`/analytics/processed/test/events` as t where dir0 = '2014'; I tried to run the following query: - select count(*) from dfs.tmp.`/analytics/processed/

Re: A incomplete map to a maze - or - the jurney down the rabbit hole

The second query, which has an expression involving UDF in the where clause, is a bit more complex than what we tested. I agree that in general partition pruning should work for expression involving functions. I posted your second query to DRILL-3560, which you opened days ago for the partition p