That didn't paste well. Here is a gist: https://gist.github.com/berryma4/c6c09da050f273295edd23c045c63403
On Mon, Jul 29, 2024 at 4:58 PM Eric Berryman <[email protected]> wrote: > I’m back from a long holiday, and seem stuck still on this scenario. The > plan gets created without any issues, but I get this error while executing > queries with unions. > > Thank you again! > Eric > > Here is the plan dump and error after running relRunner.prepareStatement. > > EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 30.0, cumulative > cost = {146.0 rows, 1393.2 cpu, 0.0 io}, id = 213 > EnumerableLimit(fetch=[100]): rowcount = 30.0, cumulative cost = {116.0 > rows, 1333.2 cpu, 0.0 io}, id = 212 EnumerableUnion(all=[true]): rowcount = > 30.0, cumulative cost = {86.0 rows, 1303.2 cpu, 0.0 io}, id = 211 > EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative > cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 207 LDAPToEnumerableConverter: > rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 206 > LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative > cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 205 > LDAPTableScan(table=[[SCIMUserSchema, cap_internet_ddpmildap]]): rowcount = > 100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id = > 0 EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative > cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 210 LDAPToEnumerableConverter: > rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 209 > LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative > cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 208 > LDAPTableScan(table=[[SCIMUserSchema, prov_internet_ddpmildap]]): rowcount > = 100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id > = 2 > > java.lang.AssertionError: null at > org.apache.calcite.adapter.enumerable.EnumerableLimit.<init>(EnumerableLimit.java:60) > at > org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:84) > at > org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:43) > at > org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:378) > at > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) at > org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at > org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286) > at > org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:228) > at > org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:514) > at > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) at > org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at > org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286) > at > org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:173) > at > org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:567) > at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:386) at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1061) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:1026) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:687) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:217) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.lambda$unwrap$0(CalciteConnectionImpl.java:187) > > > On Fri, Jul 19, 2024 at 11:07 Ruben Q L <[email protected]> wrote: > >> Eric, if you want to use EnumerableUnion instead of implementing your own >> Union, then I think you'll need to implement the appropriate Converter [1] >> to transform between conventions LDAP <=> ENUMERABLE >> >> Best, >> Ruben >> >> [1] >> >> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java >> >> >> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman <[email protected]> >> wrote: >> >> > I implement an LDAPUnion and rule to convert from logicalunion, and of >> > course, the error goes away. But I didn’t implement the union >> correctly, so >> > nothing is returned in my query. Is there a way to have my LDAPUnion >> > use EnumerableUnion, >> > or a way to not have LDAPUnion and take care of this with a rule? >> > >> > Thank you again! >> > Eric >> > >> > On Tue, Jul 16, 2024 at 14:24 Eric Berryman <[email protected]> >> > wrote: >> > >> > > Well, I was wrong. That didn’t solve my problem with : >> > > >> > > Missing conversion is LogicalUnion[convention: NONE -> LDAP] >> > > >> > > Do I have to implement my own union? Is there a way I could just use >> the >> > > EnumerableUnion? >> > > ie. NONE -> Enumerable and skip implementing one for my ldap >> datastore? >> > > >> > > Thank you! >> > > Eric >> > > >> > > On Tue, Jul 16, 2024 at 09:00 Eric Berryman <[email protected]> >> > > wrote: >> > > >> > >> When I register my tablescan object, and add my rules, I also added a >> > >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now >> > >> everything works with an offset also. >> > >> >> > >> Although, I don’t know why this is. I noticed the >> EnumerableMergeUnion >> > >> object in the plan, and thought I would try to remove it, because it >> was >> > >> different. >> > >> >> > >> Any explanation is appreciated, thank you!!! >> > >> Eric >> > >> >> > >> >> > >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman <[email protected] >> > >> > >> wrote: >> > >> >> > >>> Hello! >> > >>> >> > >>> I seem to have an issue with my new limit rule which pushes down to >> the >> > >>> datastore. It works fine, unless I add an offset to the fetch. >> Where I >> > end >> > >>> up with the following error: >> > >>> There are not enough rules… Missing conversion is >> > >>> LogicalUnion[convention: NONE -> LDAP] >> > >>> >> > >>> Why would this only come up when an offset value is added? >> > >>> >> > >>> Thank you! >> > >>> Eric >> > >>> >> > >>> >> > >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman < >> [email protected]> >> > >>> wrote: >> > >>> >> > >>>> That’s perfect >> > >>>> >> > >>>> >> > >> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 >> > >>>> >> > >>>> Thank you for such a quick response! >> > >>>> Eric >> > >>>> >> > >>>> >> > >>>> On Fri, Jun 21, 2024 at 09:14 Michael Mior <[email protected]> >> wrote: >> > >>>> >> > >>>>> Eric, >> > >>>>> >> > >>>>> Could you give a more specific example of the failure scenario >> you're >> > >>>>> experiencing? >> > >>>>> >> > >>>>> For a simple example of how limits can be pushed down, this is >> done >> > in >> > >>>>> the >> > >>>>> Cassandra adapter with CassandraLimitRule. It matches an >> > >>>>> EnumerableLimit on >> > >>>>> top of a CassandraToEnumerableConverter and then converts that >> limit >> > >>>>> to a >> > >>>>> CassandraLimit which passes along the limit and offset >> information to >> > >>>>> CassandraToEnumerableConverter for when the query is executed. The >> > >>>>> EnumerableLimit is then replaced with a CassandraLimit effectively >> > as a >> > >>>>> placeholder to signal that the limit has been handled. >> > >>>>> >> > >>>>> -- >> > >>>>> Michael Mior >> > >>>>> [email protected] >> > >>>>> >> > >>>>> >> > >>>>> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman < >> > [email protected] >> > >>>>> > >> > >>>>> wrote: >> > >>>>> >> > >>>>> > Hello! >> > >>>>> > >> > >>>>> > When I add limit to my relbuilder object, the planner gives up. >> But >> > >>>>> works >> > >>>>> > fine without it. >> > >>>>> > >> > >>>>> > I wasn’t able to find any examples of using limit and pushing >> the >> > >>>>> limit >> > >>>>> > values down to a data source. >> > >>>>> > >> > >>>>> > Could someone help with some links on this subject? >> > >>>>> > >> > >>>>> > Thank you! >> > >>>>> > Eric >> > >>>>> > >> > >>>>> >> > >>>> >> > >> >
