Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-05-04 Thread Adam Gilmore
No worries - just posted it to https://reviews.apache.org/r/33836/ and updated the JIRA accordingly. On Mon, May 4, 2015 at 11:26 PM, Jacques Nadeau wrote: > Hey Adam, just saw that. Can you post to reviewboard? If you don't have > an account, you can easily register for one. > On May 4, 2015

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-05-04 Thread Jacques Nadeau
Hey Adam, just saw that. Can you post to reviewboard? If you don't have an account, you can easily register for one. On May 4, 2015 8:25 AM, "Adam Gilmore" wrote: > Just wanted to let you know I've uploaded the patch: > > https://issues.apache.org/jira/browse/DRILL-1950 > > Be great to get som

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-05-04 Thread Adam Gilmore
Just wanted to let you know I've uploaded the patch: https://issues.apache.org/jira/browse/DRILL-1950 Be great to get some feedback or start a review board so I can see what needs to be done to get it merged in! On Fri, May 1, 2015 at 12:13 PM, Jacques Nadeau wrote: > Yes. Once per query is f

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-30 Thread Jacques Nadeau
Yes. Once per query is fine. Theoretically we could fire the rule many times inside a single query and I was saying we preferably cache in the context of a single query. On Thu, Apr 30, 2015 at 5:57 PM, Adam Gilmore wrote: > We'll need to look it up at least once per query, though, right? Bec

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-30 Thread Adam Gilmore
We'll need to look it up at least once per query, though, right? Because session variables can change query to query. Would you suggest I cached the actual setting(s) as soon as the rule is instantiated? The rule is only called once or twice per query, I think. On Fri, May 1, 2015 at 10:54 AM,

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-30 Thread Jacques Nadeau
Creating it at rule creation time is good. The key is we want to cache it so we don't have to look it up every single time the rule fires. I think your point about specific versus general and PlannerSettings makes sense. On Thu, Apr 30, 2015 at 5:17 PM, Adam Gilmore wrote: > Moreover, I'm a ta

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-30 Thread Adam Gilmore
Moreover, I'm a tad wary about using PlannerSettings as they seem very generic (i.e. not specific to a particular storage/format plugin, for example). I've used "store.parquet.enable_pushdown_filter" as the option, as it's very specific to the Parquet format plugin. I imagine I can still use getO

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-30 Thread Adam Gilmore
I actually patched the StorageEnginePlugin and FormatPlugin to pass QueryContext right through the chain of getting optimizer rules (as is the case for getting basic rules, etc.). This seemed to me like the most logical approach and aligned with something like your PruneScanRule. Do you think I s

Re: Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-29 Thread Jacques Nadeau
PlannerSettings is the primary we expose settings that need to be interrogated during planning (e.g. in an optimizer rule). You can get ahold of this by doing: PlannerSettings settings = PrelUtil.getPlannerSettings(call.getPlanner()); PlannerSettings then has access to session settings. You can

Accessing QueryContext inside a StoragePluginOptimizerRule

2015-04-29 Thread Adam Gilmore
Hi guys, I'm trying to work out how I could access the QueryContext inside a StoragePluginOptimizerRule. I've basically implemented the Parquet pushdown filtering, but I really need to access the session settings (for whether or not we're using the new Parquet reader so we can completely pushdown