Excellent point Julian, and thank you for the swift reply. We'll keep tinkering in this space to see what we can find.
Thank you, Austin On Fri, 4 Oct 2024 at 00:47, Julian Hyde <jh...@apache.org> wrote: > No, that would not be acceptable. EnumerableInterpretable#getBindable > is a package-protected method with no javadoc or tests. If you "relax" > the visibility modifier it becomes a public API and Calcite committers > are responsible for maintaining its behavior in perpetuity. > > You need to come up with a better proposal that is a net benefit to > Calcite. > > > On Thu, Oct 3, 2024 at 5:54 AM Austin Richardson > <austin.richard...@teampicnic.com.invalid> wrote: > > > > Hi Stamatis, > > > > Thank you for the reply. > > > > For some additional context: we're experimenting with this > > (de)serialisation because we're interested in storing optimised query > plans > > in a distributed cache. Another idea we're exploring is caching the > > generated source code for those query plans, and then compiling it > > as-needed later. If possible, we'd like to re-use the Calcite code > > responsible for this, though we'd need to relax the visibility modifier > of > > EnumerableInterpretable#getBindable > > < > https://github.com/apache/calcite/blob/b1308feff49c8b747b3bbb52e1519d334bc984ec/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpretable.java#L134 > > > > from > > package-private to public so it'd be invokable from our code. > > > > Do you have any thoughts on whether that'd be acceptable? > > > > Thank you, > > Austin > > > > On Tue, 1 Oct 2024 at 09:10, Stamatis Zampetakis <zabe...@gmail.com> > wrote: > > > > > Hey Austin, > > > > > > Indeed the most popular serialization/deserialization method for query > > > plans is by using the RelJsonReader and RelJsonWriter. However, we are > > > lacking a bit of test coverage around this feature so when new > > > RelNodes are added it's easy to miss the required changes. Moreover, > > > it's quite common for other projects to plug-in their own custom > > > RelNode's so serialization happens outside the classes present in > > > calcite core. We would happily accept contributions in this area so > > > feel free to create a JIRA ticket and submit a PR. > > > > > > Best, > > > Stamatis > > > > > > On Mon, Sep 30, 2024 at 11:55 AM Austin Richardson > > > <austin.richard...@teampicnic.com.invalid> wrote: > > > > > > > > Hi Calcite devs, > > > > > > > > Our team is currently experimenting with ideas for serialising and > > > > deserialising optimised query plans using RelJsonWriter and > > > RelJsonReader. > > > > We noticed this is possible for pre-optimisation logical plans, but > ran > > > > into issues when trying to do so for post-optimisation execution > plans. > > > For > > > > example, LogicalJoin implements the required constructor for > > > serialisation > > > > here > > > > < > > > > https://calcite.apache.org/javadocAggregate/org/apache/calcite/rel/logical/LogicalJoin.html#%3Cinit%3E(org.apache.calcite.rel.RelInput) > > > >, > > > > but the specific JOIN implementations (e.g. > > > EnumerableBatchNestedLoopJoin) > > > > do not. > > > > > > > > Considering this, is there a preferred way for serialising and > > > > deserialising optimised query plans? We wanted to ensure we aren’t > > > missing > > > > something here. > > > > > > > > Thank you, > > > > > > > > Austin > > > >
