​Thanks a lot everyone. Aman, your answer is very convincing. You made it clear that since a single query can involve multiple plugins, then all rules provided by at least the involved plugins must be considered by the planner.
-- Gelbana On Mon, Jul 3, 2017 at 4:04 AM, Aman Sinha <amansi...@apache.org> wrote: > Agree with Ted and Julian's comments and would add one more point: The > Planner registers the Storage plugin optimizer rules from all the plugins > [1]. The assumption is that a single query could be querying multiple data > sources, joining them in Drill etc, so it is the rule's responsibility to > have the proper constructor and matches() method to specialize it. For > example, if you have a logical planning rule and have a DrillScanRel which > is supposed to have a GroupScan of JDBCGroupScan, you could check that in > your matches() and return False if otherwise. > > > [1] > https://github.com/apache/drill/blob/master/exec/java- > exec/src/main/java/org/apache/drill/exec/planner/PlannerPhase.java#L197 > > Aman > > On Sun, Jul 2, 2017 at 11:47 AM, Julian Hyde <jh...@apache.org> wrote: > > > What Ted said. > > > > But also, conversely, you should know that in Calcite you can write a > > general-purpose rule. Or better, re-use a general-purpose rule that > someone > > else has written. There are logical rules, for example constant reduction > > and logic simplification, that work regardless of the data source. And > > there are common patterns, for example pushing down projects and filters, > > that apply to many different data sources. You would want to push down > > projects and filters to Parquet just as would would want to push them > into > > a JDBC source, albeit that Parquet cannot handle as rich an expression > > language. > > > > General-purpose rules often take classes as constructor parameters, so > you > > can instantiate the rule to look for a FooProject.class rather than a > > JdbcProject.class or LogicalProject.class. > > > > Julian > > > > > > > On Jul 2, 2017, at 11:08 AM, Ted Dunning <ted.dunn...@gmail.com> > wrote: > > > > > > It all depends on how you write your rules. If you write them so that > > they > > > apply too generally, then the rules themselves are at fault. > > > > > > If you write rules that only optimize for your input format, then you > > > should be fine. > > > > > > > > > On Jul 2, 2017 9:41 AM, "Muhammad Gelbana" <m.gelb...@gmail.com> > wrote: > > > > > >> I wrote a plugin for a custom JDBC datasource. This plugin registers a > > >> couple of rules. > > >> > > >> When I ran an SQL query that uses parquet files, I found that a rule > of > > my > > >> JDBC plugin was invoked to optimize the query ! > > >> > > >> I believe this is a mistake. Please correct me if I'm wrong. > > >> > > >> I'm saying this is a mistake because a rule registered by a plugin > that > > >> utilizes a specific datasource should only be concerned about queries > > >> executed by that plugin. > > >> > > >> Query optimizations for a JDBC datasource won't probably work for > > queries > > >> targeted for parquet files ! > > >> > > >> What do you think ? > > >> > > >> Gelbana > > >> > > > > >
