Re: Sortlimit and limit examples
That didn't paste well. Here is a gist:
https://gist.github.com/berryma4/c6c09da050f273295edd23c045c63403
On Mon, Jul 29, 2024 at 4:58 PM Eric Berryman
wrote:
> I’m back from a long holiday, and seem stuck still on this scenario. The
> plan gets created without any issues, but I get this error while executing
> queries with unions.
>
> Thank you again!
> Eric
>
> Here is the plan dump and error after running relRunner.prepareStatement.
>
> EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 30.0, cumulative
> cost = {146.0 rows, 1393.2 cpu, 0.0 io}, id = 213
> EnumerableLimit(fetch=[100]): rowcount = 30.0, cumulative cost = {116.0
> rows, 1333.2 cpu, 0.0 io}, id = 212 EnumerableUnion(all=[true]): rowcount =
> 30.0, cumulative cost = {86.0 rows, 1303.2 cpu, 0.0 io}, id = 211
> EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
> cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 207 LDAPToEnumerableConverter:
> rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 206
> LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
> cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 205
> LDAPTableScan(table=[[SCIMUserSchema, cap_internet_ddpmildap]]): rowcount =
> 100.0, cumulative cost = {10.0 rows, 10.101 cpu, 0.0 io}, id =
> 0 EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
> cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 210 LDAPToEnumerableConverter:
> rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 209
> LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
> cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 208
> LDAPTableScan(table=[[SCIMUserSchema, prov_internet_ddpmildap]]): rowcount
> = 100.0, cumulative cost = {10.0 rows, 10.101 cpu, 0.0 io}, id
> = 2
>
> java.lang.AssertionError: null at
> org.apache.calcite.adapter.enumerable.EnumerableLimit.(EnumerableLimit.java:60)
> at
> org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:84)
> at
> org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:43)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:378)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:228)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:514)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
> org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:173)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:567)
> at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:386) at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1061)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:1026)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:687)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:217)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.lambda$unwrap$0(CalciteConnectionImpl.java:187)
>
>
> On Fri, Jul 19, 2024 at 11:07 Ruben Q L wrote:
>
>> Eric, if you want to use EnumerableUnion instead of implementing your own
>> Union, then I think you'll need to implement the appropriate Converter [1]
>> to transform between conventions LDAP <=> ENUMERABLE
>>
>> Best,
>> Ruben
>>
>> [1]
>>
>> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java
>>
>>
>> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman
>> wrote:
>>
>> > I implement an LDAPUnion and rule to convert from logicalunion, and of
>> > course, the error goes away. But I didn’t implement the union
>> correctly, so
>> > nothing is returned in my query. Is there a way to have my LDAPUnion
>> > use EnumerableUnion,
>> > or a way to not have LDAPUnion and take care of this with a rule?
>> >
>> > Thank you again!
>> > Eric
>> >
>> > On Tue, Jul 16, 2024 at
Re: Sortlimit and limit examples
I’m back from a long holiday, and seem stuck still on this scenario. The
plan gets created without any issues, but I get this error while executing
queries with unions.
Thank you again!
Eric
Here is the plan dump and error after running relRunner.prepareStatement.
EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 30.0, cumulative
cost = {146.0 rows, 1393.2 cpu, 0.0 io}, id = 213
EnumerableLimit(fetch=[100]): rowcount = 30.0, cumulative cost = {116.0
rows, 1333.2 cpu, 0.0 io}, id = 212 EnumerableUnion(all=[true]): rowcount =
30.0, cumulative cost = {86.0 rows, 1303.2 cpu, 0.0 io}, id = 211
EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 207 LDAPToEnumerableConverter:
rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 206
LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 205
LDAPTableScan(table=[[SCIMUserSchema, cap_internet_ddpmildap]]): rowcount =
100.0, cumulative cost = {10.0 rows, 10.101 cpu, 0.0 io}, id =
0 EnumerableProject(UserObject=[$0], id=[$3]): rowcount = 15.0, cumulative
cost = {28.0 rows, 636.6 cpu, 0.0 io}, id = 210 LDAPToEnumerableConverter:
rowcount = 15.0, cumulative cost = {13.0 rows, 21.6 cpu, 0.0 io}, id = 209
LDAPFilter(condition=[=(UPPER($5), 'RUDD')]): rowcount = 15.0, cumulative
cost = {11.5 rows, 20.1 cpu, 0.0 io}, id = 208
LDAPTableScan(table=[[SCIMUserSchema, prov_internet_ddpmildap]]): rowcount
= 100.0, cumulative cost = {10.0 rows, 10.101 cpu, 0.0 io}, id
= 2
java.lang.AssertionError: null at
org.apache.calcite.adapter.enumerable.EnumerableLimit.(EnumerableLimit.java:60)
at
org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:84)
at
org.apache.calcite.adapter.enumerable.EnumerableLimit.copy(EnumerableLimit.java:43)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:378)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:228)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:514)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:533) at
org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:286)
at
org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:173)
at
org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:567)
at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:386) at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1061)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:1026)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:687)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:217)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.lambda$unwrap$0(CalciteConnectionImpl.java:187)
On Fri, Jul 19, 2024 at 11:07 Ruben Q L wrote:
> Eric, if you want to use EnumerableUnion instead of implementing your own
> Union, then I think you'll need to implement the appropriate Converter [1]
> to transform between conventions LDAP <=> ENUMERABLE
>
> Best,
> Ruben
>
> [1]
>
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java
>
>
> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman
> wrote:
>
> > I implement an LDAPUnion and rule to convert from logicalunion, and of
> > course, the error goes away. But I didn’t implement the union correctly,
> so
> > nothing is returned in my query. Is there a way to have my LDAPUnion
> > use EnumerableUnion,
> > or a way to not have LDAPUnion and take care of this with a rule?
> >
> > Thank you again!
> > Eric
> >
> > On Tue, Jul 16, 2024 at 14:24 Eric Berryman
> > wrote:
> >
> > > Well, I was wrong. That didn’t solve my problem with :
> > >
> > > Missing conversion is LogicalUnion[convention: NONE -> LDAP]
> > >
> > > Do I have to implement my own union? Is there a way I could just use
> the
> > > EnumerableUnion?
> > > ie. NONE -> Enumerable and skip implementing one
Re: Sortlimit and limit examples
Is there a rule that can push a limit (Sort with 0 sort keys) through Union? Hopefully there is, and it can be adapted for any subclass of Sort and Union. That rule should create the right plan. Then we can debate whether OP needs to implement a new union operator or reuse EnumerableUnion. Julian > On Jul 19, 2024, at 08:13, Ruben Q L wrote: > > Eric, if you want to use EnumerableUnion instead of implementing your own > Union, then I think you'll need to implement the appropriate Converter [1] > to transform between conventions LDAP <=> ENUMERABLE > > Best, > Ruben > > [1] > https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java > > >> On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman >> wrote: >> >> I implement an LDAPUnion and rule to convert from logicalunion, and of >> course, the error goes away. But I didn’t implement the union correctly, so >> nothing is returned in my query. Is there a way to have my LDAPUnion >> use EnumerableUnion, >> or a way to not have LDAPUnion and take care of this with a rule? >> >> Thank you again! >> Eric >> >> On Tue, Jul 16, 2024 at 14:24 Eric Berryman >> wrote: >> >>> Well, I was wrong. That didn’t solve my problem with : >>> >>> Missing conversion is LogicalUnion[convention: NONE -> LDAP] >>> >>> Do I have to implement my own union? Is there a way I could just use the >>> EnumerableUnion? >>> ie. NONE -> Enumerable and skip implementing one for my ldap datastore? >>> >>> Thank you! >>> Eric >>> >>> On Tue, Jul 16, 2024 at 09:00 Eric Berryman >>> wrote: >>> When I register my tablescan object, and add my rules, I also added a removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now everything works with an offset also. Although, I don’t know why this is. I noticed the EnumerableMergeUnion object in the plan, and thought I would try to remove it, because it was different. Any explanation is appreciated, thank you!!! Eric On Mon, Jul 15, 2024 at 18:02 Eric Berryman wrote: > Hello! > > I seem to have an issue with my new limit rule which pushes down to the > datastore. It works fine, unless I add an offset to the fetch. Where I >> end > up with the following error: > There are not enough rules… Missing conversion is > LogicalUnion[convention: NONE -> LDAP] > > Why would this only come up when an offset value is added? > > Thank you! > Eric > > > On Fri, Jun 21, 2024 at 09:25 Eric Berryman > wrote: > >> That’s perfect >> >> >> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 >> >> Thank you for such a quick response! >> Eric >> >> >> On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: >> >>> Eric, >>> >>> Could you give a more specific example of the failure scenario you're >>> experiencing? >>> >>> For a simple example of how limits can be pushed down, this is done >> in >>> the >>> Cassandra adapter with CassandraLimitRule. It matches an >>> EnumerableLimit on >>> top of a CassandraToEnumerableConverter and then converts that limit >>> to a >>> CassandraLimit which passes along the limit and offset information to >>> CassandraToEnumerableConverter for when the query is executed. The >>> EnumerableLimit is then replaced with a CassandraLimit effectively >> as a >>> placeholder to signal that the limit has been handled. >>> >>> -- >>> Michael Mior >>> mm...@apache.org >>> >>> >>> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman < >> eric.berry...@gmail.com >>> wrote: >>> Hello! When I add limit to my relbuilder object, the planner gives up. But >>> works fine without it. I wasn’t able to find any examples of using limit and pushing the >>> limit values down to a data source. Could someone help with some links on this subject? Thank you! Eric >>> >> >>
Re: Sortlimit and limit examples
Eric, if you want to use EnumerableUnion instead of implementing your own Union, then I think you'll need to implement the appropriate Converter [1] to transform between conventions LDAP <=> ENUMERABLE Best, Ruben [1] https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman wrote: > I implement an LDAPUnion and rule to convert from logicalunion, and of > course, the error goes away. But I didn’t implement the union correctly, so > nothing is returned in my query. Is there a way to have my LDAPUnion > use EnumerableUnion, > or a way to not have LDAPUnion and take care of this with a rule? > > Thank you again! > Eric > > On Tue, Jul 16, 2024 at 14:24 Eric Berryman > wrote: > > > Well, I was wrong. That didn’t solve my problem with : > > > > Missing conversion is LogicalUnion[convention: NONE -> LDAP] > > > > Do I have to implement my own union? Is there a way I could just use the > > EnumerableUnion? > > ie. NONE -> Enumerable and skip implementing one for my ldap datastore? > > > > Thank you! > > Eric > > > > On Tue, Jul 16, 2024 at 09:00 Eric Berryman > > wrote: > > > >> When I register my tablescan object, and add my rules, I also added a > >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now > >> everything works with an offset also. > >> > >> Although, I don’t know why this is. I noticed the EnumerableMergeUnion > >> object in the plan, and thought I would try to remove it, because it was > >> different. > >> > >> Any explanation is appreciated, thank you!!! > >> Eric > >> > >> > >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman > >> wrote: > >> > >>> Hello! > >>> > >>> I seem to have an issue with my new limit rule which pushes down to the > >>> datastore. It works fine, unless I add an offset to the fetch. Where I > end > >>> up with the following error: > >>> There are not enough rules… Missing conversion is > >>> LogicalUnion[convention: NONE -> LDAP] > >>> > >>> Why would this only come up when an offset value is added? > >>> > >>> Thank you! > >>> Eric > >>> > >>> > >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman > >>> wrote: > >>> > That’s perfect > > > https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 > > Thank you for such a quick response! > Eric > > > On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: > > > Eric, > > > > Could you give a more specific example of the failure scenario you're > > experiencing? > > > > For a simple example of how limits can be pushed down, this is done > in > > the > > Cassandra adapter with CassandraLimitRule. It matches an > > EnumerableLimit on > > top of a CassandraToEnumerableConverter and then converts that limit > > to a > > CassandraLimit which passes along the limit and offset information to > > CassandraToEnumerableConverter for when the query is executed. The > > EnumerableLimit is then replaced with a CassandraLimit effectively > as a > > placeholder to signal that the limit has been handled. > > > > -- > > Michael Mior > > mm...@apache.org > > > > > > On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman < > eric.berry...@gmail.com > > > > > wrote: > > > > > Hello! > > > > > > When I add limit to my relbuilder object, the planner gives up. But > > works > > > fine without it. > > > > > > I wasn’t able to find any examples of using limit and pushing the > > limit > > > values down to a data source. > > > > > > Could someone help with some links on this subject? > > > > > > Thank you! > > > Eric > > > > > > >
Re: Sortlimit and limit examples
I implement an LDAPUnion and rule to convert from logicalunion, and of course, the error goes away. But I didn’t implement the union correctly, so nothing is returned in my query. Is there a way to have my LDAPUnion use EnumerableUnion, or a way to not have LDAPUnion and take care of this with a rule? Thank you again! Eric On Tue, Jul 16, 2024 at 14:24 Eric Berryman wrote: > Well, I was wrong. That didn’t solve my problem with : > > Missing conversion is LogicalUnion[convention: NONE -> LDAP] > > Do I have to implement my own union? Is there a way I could just use the > EnumerableUnion? > ie. NONE -> Enumerable and skip implementing one for my ldap datastore? > > Thank you! > Eric > > On Tue, Jul 16, 2024 at 09:00 Eric Berryman > wrote: > >> When I register my tablescan object, and add my rules, I also added a >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now >> everything works with an offset also. >> >> Although, I don’t know why this is. I noticed the EnumerableMergeUnion >> object in the plan, and thought I would try to remove it, because it was >> different. >> >> Any explanation is appreciated, thank you!!! >> Eric >> >> >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman >> wrote: >> >>> Hello! >>> >>> I seem to have an issue with my new limit rule which pushes down to the >>> datastore. It works fine, unless I add an offset to the fetch. Where I end >>> up with the following error: >>> There are not enough rules… Missing conversion is >>> LogicalUnion[convention: NONE -> LDAP] >>> >>> Why would this only come up when an offset value is added? >>> >>> Thank you! >>> Eric >>> >>> >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman >>> wrote: >>> That’s perfect https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 Thank you for such a quick response! Eric On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: > Eric, > > Could you give a more specific example of the failure scenario you're > experiencing? > > For a simple example of how limits can be pushed down, this is done in > the > Cassandra adapter with CassandraLimitRule. It matches an > EnumerableLimit on > top of a CassandraToEnumerableConverter and then converts that limit > to a > CassandraLimit which passes along the limit and offset information to > CassandraToEnumerableConverter for when the query is executed. The > EnumerableLimit is then replaced with a CassandraLimit effectively as a > placeholder to signal that the limit has been handled. > > -- > Michael Mior > mm...@apache.org > > > On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman > > wrote: > > > Hello! > > > > When I add limit to my relbuilder object, the planner gives up. But > works > > fine without it. > > > > I wasn’t able to find any examples of using limit and pushing the > limit > > values down to a data source. > > > > Could someone help with some links on this subject? > > > > Thank you! > > Eric > > >
Re: Sortlimit and limit examples
Well, I was wrong. That didn’t solve my problem with : Missing conversion is LogicalUnion[convention: NONE -> LDAP] Do I have to implement my own union? Is there a way I could just use the EnumerableUnion? ie. NONE -> Enumerable and skip implementing one for my ldap datastore? Thank you! Eric On Tue, Jul 16, 2024 at 09:00 Eric Berryman wrote: > When I register my tablescan object, and add my rules, I also added a > removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now > everything works with an offset also. > > Although, I don’t know why this is. I noticed the EnumerableMergeUnion > object in the plan, and thought I would try to remove it, because it was > different. > > Any explanation is appreciated, thank you!!! > Eric > > > On Mon, Jul 15, 2024 at 18:02 Eric Berryman > wrote: > >> Hello! >> >> I seem to have an issue with my new limit rule which pushes down to the >> datastore. It works fine, unless I add an offset to the fetch. Where I end >> up with the following error: >> There are not enough rules… Missing conversion is >> LogicalUnion[convention: NONE -> LDAP] >> >> Why would this only come up when an offset value is added? >> >> Thank you! >> Eric >> >> >> On Fri, Jun 21, 2024 at 09:25 Eric Berryman >> wrote: >> >>> That’s perfect >>> >>> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 >>> >>> Thank you for such a quick response! >>> Eric >>> >>> >>> On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: >>> Eric, Could you give a more specific example of the failure scenario you're experiencing? For a simple example of how limits can be pushed down, this is done in the Cassandra adapter with CassandraLimitRule. It matches an EnumerableLimit on top of a CassandraToEnumerableConverter and then converts that limit to a CassandraLimit which passes along the limit and offset information to CassandraToEnumerableConverter for when the query is executed. The EnumerableLimit is then replaced with a CassandraLimit effectively as a placeholder to signal that the limit has been handled. -- Michael Mior mm...@apache.org On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman wrote: > Hello! > > When I add limit to my relbuilder object, the planner gives up. But works > fine without it. > > I wasn’t able to find any examples of using limit and pushing the limit > values down to a data source. > > Could someone help with some links on this subject? > > Thank you! > Eric > >>>
Re: Sortlimit and limit examples
When I register my tablescan object, and add my rules, I also added a removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now everything works with an offset also. Although, I don’t know why this is. I noticed the EnumerableMergeUnion object in the plan, and thought I would try to remove it, because it was different. Any explanation is appreciated, thank you!!! Eric On Mon, Jul 15, 2024 at 18:02 Eric Berryman wrote: > Hello! > > I seem to have an issue with my new limit rule which pushes down to the > datastore. It works fine, unless I add an offset to the fetch. Where I end > up with the following error: > There are not enough rules… Missing conversion is LogicalUnion[convention: > NONE -> LDAP] > > Why would this only come up when an offset value is added? > > Thank you! > Eric > > > On Fri, Jun 21, 2024 at 09:25 Eric Berryman > wrote: > >> That’s perfect >> >> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 >> >> Thank you for such a quick response! >> Eric >> >> >> On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: >> >>> Eric, >>> >>> Could you give a more specific example of the failure scenario you're >>> experiencing? >>> >>> For a simple example of how limits can be pushed down, this is done in >>> the >>> Cassandra adapter with CassandraLimitRule. It matches an EnumerableLimit >>> on >>> top of a CassandraToEnumerableConverter and then converts that limit to a >>> CassandraLimit which passes along the limit and offset information to >>> CassandraToEnumerableConverter for when the query is executed. The >>> EnumerableLimit is then replaced with a CassandraLimit effectively as a >>> placeholder to signal that the limit has been handled. >>> >>> -- >>> Michael Mior >>> mm...@apache.org >>> >>> >>> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman >>> wrote: >>> >>> > Hello! >>> > >>> > When I add limit to my relbuilder object, the planner gives up. But >>> works >>> > fine without it. >>> > >>> > I wasn’t able to find any examples of using limit and pushing the limit >>> > values down to a data source. >>> > >>> > Could someone help with some links on this subject? >>> > >>> > Thank you! >>> > Eric >>> > >>> >>
Re: Sortlimit and limit examples
Hello! I seem to have an issue with my new limit rule which pushes down to the datastore. It works fine, unless I add an offset to the fetch. Where I end up with the following error: There are not enough rules… Missing conversion is LogicalUnion[convention: NONE -> LDAP] Why would this only come up when an offset value is added? Thank you! Eric On Fri, Jun 21, 2024 at 09:25 Eric Berryman wrote: > That’s perfect > > https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 > > Thank you for such a quick response! > Eric > > > On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: > >> Eric, >> >> Could you give a more specific example of the failure scenario you're >> experiencing? >> >> For a simple example of how limits can be pushed down, this is done in the >> Cassandra adapter with CassandraLimitRule. It matches an EnumerableLimit >> on >> top of a CassandraToEnumerableConverter and then converts that limit to a >> CassandraLimit which passes along the limit and offset information to >> CassandraToEnumerableConverter for when the query is executed. The >> EnumerableLimit is then replaced with a CassandraLimit effectively as a >> placeholder to signal that the limit has been handled. >> >> -- >> Michael Mior >> mm...@apache.org >> >> >> On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman >> wrote: >> >> > Hello! >> > >> > When I add limit to my relbuilder object, the planner gives up. But >> works >> > fine without it. >> > >> > I wasn’t able to find any examples of using limit and pushing the limit >> > values down to a data source. >> > >> > Could someone help with some links on this subject? >> > >> > Thank you! >> > Eric >> > >> >
Re: Sortlimit and limit examples
That’s perfect https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401 Thank you for such a quick response! Eric On Fri, Jun 21, 2024 at 09:14 Michael Mior wrote: > Eric, > > Could you give a more specific example of the failure scenario you're > experiencing? > > For a simple example of how limits can be pushed down, this is done in the > Cassandra adapter with CassandraLimitRule. It matches an EnumerableLimit on > top of a CassandraToEnumerableConverter and then converts that limit to a > CassandraLimit which passes along the limit and offset information to > CassandraToEnumerableConverter for when the query is executed. The > EnumerableLimit is then replaced with a CassandraLimit effectively as a > placeholder to signal that the limit has been handled. > > -- > Michael Mior > mm...@apache.org > > > On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman > wrote: > > > Hello! > > > > When I add limit to my relbuilder object, the planner gives up. But works > > fine without it. > > > > I wasn’t able to find any examples of using limit and pushing the limit > > values down to a data source. > > > > Could someone help with some links on this subject? > > > > Thank you! > > Eric > > >
Re: Sortlimit and limit examples
Eric, Could you give a more specific example of the failure scenario you're experiencing? For a simple example of how limits can be pushed down, this is done in the Cassandra adapter with CassandraLimitRule. It matches an EnumerableLimit on top of a CassandraToEnumerableConverter and then converts that limit to a CassandraLimit which passes along the limit and offset information to CassandraToEnumerableConverter for when the query is executed. The EnumerableLimit is then replaced with a CassandraLimit effectively as a placeholder to signal that the limit has been handled. -- Michael Mior mm...@apache.org On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman wrote: > Hello! > > When I add limit to my relbuilder object, the planner gives up. But works > fine without it. > > I wasn’t able to find any examples of using limit and pushing the limit > values down to a data source. > > Could someone help with some links on this subject? > > Thank you! > Eric >
Sortlimit and limit examples
Hello! When I add limit to my relbuilder object, the planner gives up. But works fine without it. I wasn’t able to find any examples of using limit and pushing the limit values down to a data source. Could someone help with some links on this subject? Thank you! Eric
