Hi Lee, Welcome to Drill!
Yes, Drill does allow the storage/format plugins to inspect the WHERE clause and modify the query plan using optimizer rules. Please take a look at HBase storage plugin for similar example The storage plugin needs to specify the set of optimizer rule[1] and then parse and modify[2] the query plan in the optimizer rule. aditya... [1] https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePlugin.java#L80 [2] https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBasePushFilterIntoScan.java On Tue, Mar 22, 2016 at 7:59 PM, Lee Butts <[email protected]> wrote: > Hi, > > I'm having a go at adding a format plugin to Drill and would like to be > able to access the where clause of a running query in order to skip through > a large data file (currently our data files are in 1 hour chunks and can be > hundreds of MBs). > > In my use case the where clause would contain a "between" clause that I > could use to skip/filter rows based on a timestamp column. > > I can't see anywhere in the various contexts that a format plugin has > access to that the query AST or something similar is held. > > Is that something that drill could ever support? > > thanks > > Lee > > [image: MahiFX] <https://mahifx.com/> > > Lee Butts > Technology > Level 3, 50 Victoria Street > Christchurch 8013 > New Zealand > Mobile: +64 226 908315 > Skype: leemahifx > > <https://www.facebook.com/mahifx> <https://twitter.com/mahiforex> > <https://www.youtube.com/channel/UCxCiKv8XEM8GATA8kFFqsAw> > <https://www.linkedin.com/company/mahifx> > <https://mahifx.com/blog/feed.xml> < > https://plus.google.com/+Mahifx/posts> > > > [image: MFX Compass] <http://mfxcompass.mahifx.com/> > > -- > > > ------------------------------ > > *IMPORTANT NOTICE*: MahiFX Limited (*MahiFX*), (company no. 2446590) is > registered in New Zealand. MahiFX is authorised and regulated by the > Australian Securities and Investment Commission ARBN 152 535 085, AFSL > 414198. This email, its attachments and any rights attaching hereto are > confidential and intended exclusively for the person to whom the email is > addressed. If you are not the intended recipient, do not read, copy, > disclose or use the contents in any way. Please notify the sender by return > email and destroy the email and attachments immediately. MahiFX does not > accept any liability for any changes made to this email or attachments > after sending by MahiFX. You must scan this email and attachments for > viruses. The opinions expressed are not necessarily those of MahiFX. MahiFX > accepts no liability for any loss, damage or consequence, whether caused by > our own negligence or not, resulting directly or indirectly from the use of > this email and attachments. > > For more information about MahiFX Limited see mahifx.com. >
