I’m back from a long holiday, and seem stuck still on this scenario. The
plan gets created without any issues, but I get this error while executing
queries with unions.
Thank you again!
Eric
Here is the plan dump and error after running relRunner.prepareStatement.
EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 30.0, cumulative
cost = {146.0 rows, 1393.2 cpu, 0.0 io}, id = 213
EnumerableLimit(fetch=[100]): rowcount = 30.0, cumulative cost = {116.0
rows, 1333.2 cpu, 0.0 io}, id = 212 EnumerableUnion(all=[true]): rowcount =
30.0, cumulative cost = {86.0 rows, 1303.2 cpu, 0.0 io}, id = 211
EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 207 LDAPToEnumerableConverter:
rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 206
LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 205
LDAPTableScan(table=[[SCIMUserSchema, cap_internet_ddpmildap]]): rowcount =
100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id =
0 EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 210 LDAPToEnumerableConverter:
rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 209
LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 208
LDAPTableScan(table=[[SCIMUserSchema, prov_internet_ddpmildap]]): rowcount
= 100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id
= 2
java.lang.AssertionError: null at
org.apache.calcite.adapter.enumerable.EnumerableLimit.<init>(EnumerableLimit.java:60)
at
org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:84)
at
org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:43)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:378)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:228)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:514)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:173)
at
org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:567)
at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:386) at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1061)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:1026)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:687)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:217)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.lambda$unwrap$0(CalciteConnectionImpl.java:187)
On Fri, Jul 19, 2024 at 11:07 Ruben Q L <rube...@gmail.com> wrote:
> Eric, if you want to use EnumerableUnion instead of implementing your own
> Union, then I think you'll need to implement the appropriate Converter [1]
> to transform between conventions LDAP <=> ENUMERABLE
>
> Best,
> Ruben
>
> [1]
>
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java
>
>
> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman <eric.berry...@gmail.com>
> wrote:
>
> > I implement an LDAPUnion and rule to convert from logicalunion, and of
> > course, the error goes away. But I didn’t implement the union correctly,
> so
> > nothing is returned in my query. Is there a way to have my LDAPUnion
> > use EnumerableUnion,
> > or a way to not have LDAPUnion and take care of this with a rule?
> >
> > Thank you again!
> > Eric
> >
> > On Tue, Jul 16, 2024 at 14:24 Eric Berryman <eric.berry...@gmail.com>
> > wrote:
> >
> > > Well, I was wrong. That didn’t solve my problem with :
> > >
> > > Missing conversion is LogicalUnion[convention: NONE -> LDAP]
> > >
> > > Do I have to implement my own union? Is there a way I could just use
> the
> > > EnumerableUnion?
> > > ie. NONE -> Enumerable and skip implementing one for my ldap datastore?
> > >
> > > Thank you!
> > > Eric
> > >
> > > On Tue, Jul 16, 2024 at 09:00 Eric Berryman <eric.berry...@gmail.com>
> > > wrote:
> > >
> > >> When I register my tablescan object, and add my rules, I also added a
> > >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now
> > >> everything works with an offset also.
> > >>
> > >> Although, I don’t know why this is. I noticed the EnumerableMergeUnion
> > >> object in the plan, and thought I would try to remove it, because it
> was
> > >> different.
> > >>
> > >> Any explanation is appreciated, thank you!!!
> > >> Eric
> > >>
> > >>
> > >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman <eric.berry...@gmail.com>
> > >> wrote:
> > >>
> > >>> Hello!
> > >>>
> > >>> I seem to have an issue with my new limit rule which pushes down to
> the
> > >>> datastore. It works fine, unless I add an offset to the fetch. Where
> I
> > end
> > >>> up with the following error:
> > >>> There are not enough rules… Missing conversion is
> > >>> LogicalUnion[convention: NONE -> LDAP]
> > >>>
> > >>> Why would this only come up when an offset value is added?
> > >>>
> > >>> Thank you!
> > >>> Eric
> > >>>
> > >>>
> > >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman <eric.berry...@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>> That’s perfect
> > >>>>
> > >>>>
> >
> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401
> > >>>>
> > >>>> Thank you for such a quick response!
> > >>>> Eric
> > >>>>
> > >>>>
> > >>>> On Fri, Jun 21, 2024 at 09:14 Michael Mior <mm...@apache.org>
> wrote:
> > >>>>
> > >>>>> Eric,
> > >>>>>
> > >>>>> Could you give a more specific example of the failure scenario
> you're
> > >>>>> experiencing?
> > >>>>>
> > >>>>> For a simple example of how limits can be pushed down, this is done
> > in
> > >>>>> the
> > >>>>> Cassandra adapter with CassandraLimitRule. It matches an
> > >>>>> EnumerableLimit on
> > >>>>> top of a CassandraToEnumerableConverter and then converts that
> limit
> > >>>>> to a
> > >>>>> CassandraLimit which passes along the limit and offset information
> to
> > >>>>> CassandraToEnumerableConverter for when the query is executed. The
> > >>>>> EnumerableLimit is then replaced with a CassandraLimit effectively
> > as a
> > >>>>> placeholder to signal that the limit has been handled.
> > >>>>>
> > >>>>> --
> > >>>>> Michael Mior
> > >>>>> mm...@apache.org
> > >>>>>
> > >>>>>
> > >>>>> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman <
> > eric.berry...@gmail.com
> > >>>>> >
> > >>>>> wrote:
> > >>>>>
> > >>>>> > Hello!
> > >>>>> >
> > >>>>> > When I add limit to my relbuilder object, the planner gives up.
> But
> > >>>>> works
> > >>>>> > fine without it.
> > >>>>> >
> > >>>>> > I wasn’t able to find any examples of using limit and pushing the
> > >>>>> limit
> > >>>>> > values down to a data source.
> > >>>>> >
> > >>>>> > Could someone help with some links on this subject?
> > >>>>> >
> > >>>>> > Thank you!
> > >>>>> > Eric
> > >>>>> >
> > >>>>>
> > >>>>
> >
>
