Re: How to use VolcanoPlanner

2016-11-01 Thread Colm McHugh
What errors did you get? It should be possible to use both Volcano and hep when query planning (Drill does this, and possibly others). It superficially sounds like applying a heuristics pass that includes the project pushdown (and any other rules you may want to apply) after the volcano pass should

Re: How to use VolcanoPlanner

2016-11-01 Thread Julian Hyde
There is a simple form of projection push-down, namely column pruning aka field trimming. If you compute x + y from a table that has columns (x, y, z), then it will prune away z, and project only (x, y). Column pruning doesn’t fit into the Volcano model (or in fact into the general transformatio

Re: How to use VolcanoPlanner

2016-11-01 Thread Γιώργος Θεοδωράκης
I fixed the errors (they occurred because of the way I added EnumerableRules in Volcano). I have implemented something like what you suggested: 1)I use VolcanoPlanner (with both simple and EnumerableRules), and get a plan with EnumerableRules. As I have found by the hard way, I cannot get a plan

Re: How to use VolcanoPlanner

2016-11-01 Thread Colm McHugh
What errors did you get? It should be possible to use both Volcano and hep when query planning (Drill does this, and possibly others). It superficially sounds like applying a heuristics pass that includes the project pushdown (and any other rules you may want to apply) after the volcano pass should

Re: How to use VolcanoPlanner

2016-11-01 Thread Γιώργος Θεοδωράκης
I am wondering if is it possible to push down projections in Volcano generally. With the cost model of Volcano, a projection adds rows and cpu cost and it can't be chosen. For example for the next query: "select s.products.productid " + "from s.products,s.orders " + "where s.orders.productid = s.

Re: How to use VolcanoPlanner

2016-10-27 Thread Γιώργος Θεοδωράκης
I fixed the second error by changing my Statistics to: public Statistic getStatistic() { int rowCount = rows.size(); return Statistics.of(rowCount, ImmutableList.of()); } Any suggestions of a better solution are welcome. The project push-down problem still exists... 2016-10-27 15:42 GMT+03:00 Γι

Re: How to use VolcanoPlanner

2016-10-27 Thread Γιώργος Θεοδωράκης
Hi, I was missing the implementations of operators, and I added the built in EnumerableRules until I create my own, in order to fix it. However, the plan I get from Volcano Optimizer is different from the one I get from HepPlanner, although I use the same rules. My problem is about Projection push-

Re: How to use VolcanoPlanner

2016-10-15 Thread Jungtaek Lim
Hi George, This patch is ported version (with small fixes) of Milinda's samza-sql implementation for Storm SQL. https://github.com/apache/storm/pull/1736 In this patch I removed adding HepPlanner and just rely on Volcano Planner (so the patch may be the closer thing what you want). For now I also

Re: How to use VolcanoPlanner

2016-10-04 Thread Γιώργος Θεοδωράκης
I think I did as you said: https://github.com/giwrgostheod/Calcite-Saber/blob/master/src/main/java/calcite/VolcanoTester.java and I get for every query I use: Exception in thread "main" org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node [rel#10:Subset#2.NONE.[]] could not be implement

Re: How to use VolcanoPlanner

2016-10-03 Thread Jordan Halterman
The link you provided is a pretty good example. Build a FrameworkConfig with your schema, parser config, and other information, and use it to create a Planner. That Planner uses a VolcanoPlanner internally. What's missing from that particular example is just the addition of programs. Programs are e

How to use VolcanoPlanner

2016-10-03 Thread Γιώργος Θεοδωράκης
Hi, I want to parse an Sql query and transform it to an optimized relational plan (not convert it to physical !!) using calcite rules based on my database schema and metadata. Right now, the only helpful example I have found for my purpose is taken from https://github.com/milinda/samza-sql/blob/ma