Thanks a lot Aman, my mistake was that there was a logical node between
physical ones, which couldn't be implemented of course.
- Gelbana
On Tue, Jul 11, 2017 at 5:51 PM, Aman Sinha <amansi...@apache.org> wrote:
> For such 'could not be implemented' issues, it is useful to look at Calcite
> trace logs to determine where an appropriate implementation of the logical
> plan node to physical could not be found. You would need 2 things: a
> logging.properties file and adding couple of -D flags to your JVM
> properties.
>
> In drill-env.sh:
>
> export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS *-Dcalcite.debug=true
> -Djava.util.logging.config.file=<path_to_logging.properties>* "
>
> // Create logging.properties file (note the output of this will go to your
> $HOME/log directory, so you need to create that).
>
> $ cat logging.properties
>
> handlers=java.util.logging.FileHandler
>
> .level=ALL
>
> #.level= INFO
>
> org.eigenbase.relopt.RelOptPlanner.level=FINER
>
> java.util.logging.FileHandler.pattern=%h/log/java%u.log
>
> On Mon, Jul 10, 2017 at 9:20 AM, Muhammad Gelbana <m.gelb...@gmail.com>
> wrote:
>
> > Forgive me for accidentally sending the previous email before preparing
> it
> > well. Ignore the plans mentioned earlier for now, to continue...
> >
> > Getting back to the rules, I edited the old JDBC physical converter rule
> to
> > be
> >
> > private static class GelbanaPrule extends ConverterRule {
> > > private GelbanaPrule(IncortaLayoutConvention
> > incortaLayoutConvention)
> > > {
> > > super(GelbanaRel.class, incortaLayoutConvention,
> > > Prel.DRILL_PHYSICAL, "PREL_Converter");
> > > }
> > > @Override
> > > public boolean matches(RelOptRuleCall call) {
> > > return super.matches(call);
> > > }
> > > @Override
> > > public RelNode convert(RelNode in) {
> > > RelTraitSet physicalTraits =
> > > in.getTraitSet().replace(getOutTrait());
> > > RelTraitSet noneTraits = in.getTraitSet().replace(
> > Convention.NONE);
> > > return new GelbanaIntermediatePrel(in.getCluster(),
> > > physicalTraits, convert(in, noneTraits));
> > > }
> > > }
> >
> >
> > What happens is that the physical rule is executed successfully but then
> an
> > error (*Node [rel#50:Subset#3.LOGICAL.ANY([]).[]] could not be
> > implemented;
> > planner state*) is thrown from this method:
> > *org.apache.calcite.plan.volcano.RelSubset.CheapestPlanReplacer.visit(
> > RelNode,
> > int, RelNode) *
> >
> > That's because a *RelSubset* is visited but it doesn't have a best
> > performing node and it's cost is infinite.
> >
> > Getting back to the plan included in the previous email, I encapsulated
> the
> > *LogicalAggregate* as an *IncortaRel*, so that I can physically implement
> > the aggregation. I succeeded in the encapsulation but I can't figure out
> > how to fix the "*could not be implemented*" error so far. Would someone
> > please give a hint about how I can approach this error ?
> >
> >
> >
> > - Gelbana
> >
> > On Mon, Jul 10, 2017 at 6:09 PM, Muhammad Gelbana <m.gelb...@gmail.com>
> > wrote:
> >
> > > I'm planning to create a storage plugin for a custom datasource that
> > > accepts queries in the form of XML messages, but it's metadata can be
> > > discovered using the JDBC metadata API. I can work on discovering the
> > > metadata differently but that's not my priority for now.
> > >
> > > So I copied the JDBC storage plugin, Ignored all JDBC rules, and edited
> > > wrote the following JDBC storage rules:
> > >
> > > I renamed *JdbcDrelConverterRule* to *GelbanaRelConverterRule* and
> edited
> > > it's *constructor* and *convert* methods to be:
> > >
> > > public GelbanaRelConverterRule(IncortaLayoutConvention out) {
> > >> super(Aggregate.class, Convention.NONE, out,
> > >> "Incorta_Rel_Converter");
> > >> }
> > >> @Override
> > >> public RelNode convert(RelNode rel) {
> > >> RelTraitSet newTraits = rel.getTraitSet().replace(getOutTrait());
> > >> return new GelbanaRel(rel.getCluster(), newTraits, convert(rel,
> > >> newTraits));
> > >> }
> > >
> > >
> > > 17:57:19.931 [269c5c27-2f94-14ff-1f3e-0035b17b5965:foreman] DEBUG
> > > o.a.d.e.p.s.h.DefaultSqlHandler - HEP:Window Function rewrites
> (152ms):
> > > LogicalProject(EXPR$0=[$1], PROD_CATEGORY=[$0]): rowcount = 150.0,
> > > cumulative cost = {3518.75 rows, 3502.0 cpu, 0.0 io, 0.0 network, 0.0
> > > memory}, id = 21
> > > LogicalAggregate(group=[{0}], EXPR$0=[COUNT($1)]): rowcount = 150.0,
> > > cumulative cost = {3368.75 rows, 3202.0 cpu, 0.0 io, 0.0 network, 0.0
> > > memory}, id = 19
> > > LogicalProject(PROD_CATEGORY=[$13], Revenue=[$3]): rowcount =
> > 1500.0,
> > > cumulative cost = {3200.0 rows, 3202.0 cpu, 0.0 io, 0.0 network, 0.0
> > > memory}, id = 17
> > > LogicalJoin(condition=[=($0, $13)], joinType=[inner]): rowcount =
> > > 1500.0, cumulative cost = {1700.0 rows, 202.0 cpu, 0.0 io, 0.0 network,
> > 0.0
> > > memory}, id = 15
> > > JdbcTableScan(table=[[incorta, SALES, Target]]): rowcount =
> > 100.0,
> > > cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0
> > memory},
> > > id = 7
> > > JdbcTableScan(table=[[incorta, SALES, PRODUCTS]]): rowcount =
> > > 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network,
> 0.0
> > > memory}, id = 8
> > >
> > > 17:57:20.094 [269c5c27-2f94-14ff-1f3e-0035b17b5965:foreman] DEBUG
> > > o.a.d.e.p.s.h.DefaultSqlHandler - HEP_BOTTOM_UP:Directory Prune
> Planning
> > > (150ms):
> > > LogicalProject(EXPR$0=[$1], PROD_CATEGORY=[$0]): rowcount = 150.0,
> > > cumulative cost = {150.0 rows, 300.0 cpu, 0.0 io, 0.0 network, 0.0
> > memory},
> > > id = 37
> > > IncortaRel: rowcount = 150.0, cumulative cost = {0.0 rows, 0.0 cpu,
> 0.0
> > > io, 0.0 network, 0.0 memory}, id = 45
> > >
> > >
> > > - Gelbana
> > >
> >
>
