Is there anymore information that would be helpful for me to give to aide in troubleshooting this?
Thank you! Eric On Thu, Aug 15, 2024 at 16:51 Eric Berryman <[email protected]> wrote: > > I seem to be struggling with unions still. Here is a gist with the output > and svg of the plan. > > https://gist.github.com/berryma4/c4870dd4e292e0509d1c85c308b52e67 > > I need a little help on direction of what the svg is telling me. I see > the cost is infinite, but I don’t see where I’m missing a rule. > > > Thank you for your help, > Eric > > > > On Mon, Jul 29, 2024 at 18:28 Eric Berryman <[email protected]> > wrote: > >> That didn't paste well. Here is a gist: >> https://gist.github.com/berryma4/c6c09da050f273295edd23c045c63403 >> >> >> >> On Mon, Jul 29, 2024 at 4:58 PM Eric Berryman <[email protected]> >> wrote: >> >>> I’m back from a long holiday, and seem stuck still on this scenario. The >>> plan gets created without any issues, but I get this error while executing >>> queries with unions. >>> >>> Thank you again! >>> Eric >>> >>> Here is the plan dump and error after running relRunner.prepareStatement. >>> >>> EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 30.0, cumulative >>> cost = {146.0 rows, 1393.2 cpu, 0.0 io}, id = 213 >>> EnumerableLimit(fetch=[100]): rowcount = 30.0, cumulative cost = {116.0 >>> rows, 1333.2 cpu, 0.0 io}, id = 212 EnumerableUnion(all=[true]): rowcount = >>> 30.0, cumulative cost = {86.0 rows, 1303.2 cpu, 0.0 io}, id = 211 >>> EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative >>> cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 207 LDAPToEnumerableConverter: >>> rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 206 >>> LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative >>> cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 205 >>> LDAPTableScan(table=[[SCIMUserSchema, cap_internet_ddpmildap]]): rowcount = >>> 100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id = >>> 0 EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative >>> cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 210 LDAPToEnumerableConverter: >>> rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 209 >>> LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative >>> cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 208 >>> LDAPTableScan(table=[[SCIMUserSchema, prov_internet_ddpmildap]]): rowcount >>> = 100.0, cumulative cost = {10.0 rows, 10.100000000000001 cpu, 0.0 io}, id >>> = 2 >>> >>> java.lang.AssertionError: null at >>> org.apache.calcite.adapter.enumerable.EnumerableLimit.<init>(EnumerableLimit.java:60) >>> at >>> org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:84) >>> at >>> org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:43) >>> at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:378) >>> at >>> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:580) at >>> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286) >>> at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:228) >>> at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:514) >>> at >>> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:580) at >>> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286) >>> at >>> org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:173) >>> at >>> org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:567) >>> at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:386) at >>> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1061) >>> at >>> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:1026) >>> at >>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:687) >>> at >>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519) >>> at >>> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487) >>> at >>> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237) >>> at >>> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:217) >>> at >>> org.apache.calcite.jdbc.CalciteConnectionImpl.lambda$unwrap$0(CalciteConnectionImpl.java:187) >>> >>> >>> On Fri, Jul 19, 2024 at 11:07 Ruben Q L <[email protected]> wrote: >>> >>>> Eric, if you want to use EnumerableUnion instead of implementing your >>>> own >>>> Union, then I think you'll need to implement the appropriate Converter >>>> [1] >>>> to transform between conventions LDAP <=> ENUMERABLE >>>> >>>> Best, >>>> Ruben >>>> >>>> [1] >>>> >>>> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java >>>> >>>> >>>> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman <[email protected]> >>>> wrote: >>>> >>>> > I implement an LDAPUnion and rule to convert from logicalunion, and of >>>> > course, the error goes away. But I didn’t implement the union >>>> correctly, so >>>> > nothing is returned in my query. Is there a way to have my LDAPUnion >>>> > use EnumerableUnion, >>>> > or a way to not have LDAPUnion and take care of this with a rule? >>>> > >>>> > Thank you again! >>>> > Eric >>>> > >>>> > On Tue, Jul 16, 2024 at 14:24 Eric Berryman <[email protected]> >>>> > wrote: >>>> > >>>> > > Well, I was wrong. That didn’t solve my problem with : >>>> > > >>>> > > Missing conversion is LogicalUnion[convention: NONE -> LDAP] >>>> > > >>>> > > Do I have to implement my own union? Is there a way I could just >>>> use the >>>> > > EnumerableUnion? >>>> > > ie. NONE -> Enumerable and skip implementing one for my ldap >>>> datastore? >>>> > > >>>> > > Thank you! >>>> > > Eric >>>> > > >>>> > > On Tue, Jul 16, 2024 at 09:00 Eric Berryman < >>>> [email protected]> >>>> > > wrote: >>>> > > >>>> > >> When I register my tablescan object, and add my rules, I also >>>> added a >>>> > >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now >>>> > >> everything works with an offset also. >>>> > >> >>>> > >> Although, I don’t know why this is. I noticed the >>>> EnumerableMergeUnion >>>> > >> object in the plan, and thought I would try to remove it, because >>>> it was >>>> > >> different. >>>> > >> >>>> > >> Any explanation is appreciated, thank you!!! >>>> > >> Eric >>>> > >> >>>> > >> >>>> > >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman < >>>> [email protected]> >>>> > >> wrote: >>>> > >> >>>> > >>> Hello! >>>> > >>> >>>> > >>> I seem to have an issue with my new limit rule which pushes down >>>> to the >>>> > >>> datastore. It works fine, unless I add an offset to the fetch. >>>> Where I >>>> > end >>>> > >>> up with the following error: >>>> > >>> There are not enough rules… Missing conversion is >>>> > >>> LogicalUnion[convention: NONE -> LDAP] >>>> > >>> >>>> > >>> Why would this only come up when an offset value is added? >>>> > >>> >>>> > >>> Thank you! >>>> > >>> Eric >>>> > >>> >>>> > >>> >>>> > >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman < >>>> [email protected]> >>>> > >>> wrote: >>>> > >>> >>>> > >>>> That’s perfect >>>> > >>>> >>>> > >>>> >>>> > >>>> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 >>>> > >>>> >>>> > >>>> Thank you for such a quick response! >>>> > >>>> Eric >>>> > >>>> >>>> > >>>> >>>> > >>>> On Fri, Jun 21, 2024 at 09:14 Michael Mior <[email protected]> >>>> wrote: >>>> > >>>> >>>> > >>>>> Eric, >>>> > >>>>> >>>> > >>>>> Could you give a more specific example of the failure scenario >>>> you're >>>> > >>>>> experiencing? >>>> > >>>>> >>>> > >>>>> For a simple example of how limits can be pushed down, this is >>>> done >>>> > in >>>> > >>>>> the >>>> > >>>>> Cassandra adapter with CassandraLimitRule. It matches an >>>> > >>>>> EnumerableLimit on >>>> > >>>>> top of a CassandraToEnumerableConverter and then converts that >>>> limit >>>> > >>>>> to a >>>> > >>>>> CassandraLimit which passes along the limit and offset >>>> information to >>>> > >>>>> CassandraToEnumerableConverter for when the query is executed. >>>> The >>>> > >>>>> EnumerableLimit is then replaced with a CassandraLimit >>>> effectively >>>> > as a >>>> > >>>>> placeholder to signal that the limit has been handled. >>>> > >>>>> >>>> > >>>>> -- >>>> > >>>>> Michael Mior >>>> > >>>>> [email protected] >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman < >>>> > [email protected] >>>> > >>>>> > >>>> > >>>>> wrote: >>>> > >>>>> >>>> > >>>>> > Hello! >>>> > >>>>> > >>>> > >>>>> > When I add limit to my relbuilder object, the planner gives >>>> up. But >>>> > >>>>> works >>>> > >>>>> > fine without it. >>>> > >>>>> > >>>> > >>>>> > I wasn’t able to find any examples of using limit and pushing >>>> the >>>> > >>>>> limit >>>> > >>>>> > values down to a data source. >>>> > >>>>> > >>>> > >>>>> > Could someone help with some links on this subject? >>>> > >>>>> > >>>> > >>>>> > Thank you! >>>> > >>>>> > Eric >>>> > >>>>> > >>>> > >>>>> >>>> > >>>> >>>> > >>>> >>>
