So, I filed the issue here, https://issues.apache.org/jira/browse/DRILL-3692
If more details are needed let me know. Sungwook On Sun, Aug 23, 2015 at 2:45 PM, Aman Sinha <asi...@maprtech.com> wrote: > Yes, I just realized that and was about to respond to my prior message. > I just tested with a directory structure similar to Sungwook's (where > directories are named with 'year=2012' format) and it works for me. > But I am on the current master branch. > In the original message 'Sometimes it picks up e.g., year=2010, but not > year=2012..' that clearly sounds like wrong result... > definitely file a JIRA with a repro. > > Aman > > On Sun, Aug 23, 2015 at 12:23 PM, Jacques Nadeau <jacq...@dremio.com> > wrote: > > > The way that Sungwook is describing the issue, it has nothing to do with > > Hive. The files were generated via Hive but he is querying directly > > through the DFS schema. > > > > -- > > Jacques Nadeau > > CTO and Co-Founder, Dremio > > > > On Sun, Aug 23, 2015 at 12:20 PM, Aman Sinha <asi...@maprtech.com> > wrote: > > > > > Sungwook, do you have the latest master build which has the fix for > Hive > > > partition pruning (DRILL-3121) ? > > > > > > On Sun, Aug 23, 2015 at 12:15 PM, Sungwook Yoon <sy...@maprtech.com> > > > wrote: > > > > > > > Will do, > > > > > > > > Thanks, > > > > > > > > Sungwook > > > > > > > > > > > > On Sun, Aug 23, 2015 at 2:14 PM, Jacques Nadeau <jacq...@dremio.com> > > > > wrote: > > > > > > > > > It sounds like a bug. Can you file a jira? > > > > > > > > > > -- > > > > > Jacques Nadeau > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > On Sun, Aug 23, 2015 at 12:13 PM, Sungwook Yoon < > sy...@maprtech.com> > > > > > wrote: > > > > > > > > > > > Hi Jacques, > > > > > > > > > > > > This works well, no problem of accessing the partitioned dirs. > > > > > > (and actually pretty faster than accessing from one level above) > > > > > > > > > > > > Just the issues I asked about, when I access from the > > > > > > /user/hive/warehouse/table, it somehow does not recover every > dir0. > > > > > > > > > > > > Sungwook > > > > > > > > > > > > > > > > > > On Sun, Aug 23, 2015 at 2:02 PM, Jacques Nadeau < > > jacq...@dremio.com> > > > > > > wrote: > > > > > > > > > > > > > I think Hsuan misunderstood your question. > > > > > > > > > > > > > > Can you let us know what you get if you query: > > > > > > > > > > > > > > select * from dfs.root.`/user/hive/warehouse/table/year=2012` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Jacques Nadeau > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > On Sun, Aug 23, 2015 at 7:07 AM, Sungwook Yoon < > > sy...@maprtech.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I am trying to use Hive parquet stored files partitioned by > > some > > > > > > column. > > > > > > > > So, the directory structure is partitioned with the column. > > > > > > > > > > > > > > > > The column is actually year. > > > > > > > > Let's say there are 5 years, so dir0 are like year=2010, > > > > > > > > year=2011,year=2012,year=2013,year=2014 > > > > > > > > > > > > > > > > We did like following > > > > > > > > select * from dfs.root.`/user/hive/warehouse/table` d where > > > d.dir0 > > > > = > > > > > > > > 'year=2012'; > > > > > > > > > > > > > > > > I get nothing. > > > > > > > > Apparently, there are parquet files in the directory though. > > > > > > > > > > > > > > > > Sometimes it picks up e.g., year=2010, but not year=2012.. > > > > > > > > > > > > > > > > Where am I going wrong with this? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Sungwook > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
