Hi Stamatis, Thank you for the reply.
For some additional context: we're experimenting with this (de)serialisation because we're interested in storing optimised query plans in a distributed cache. Another idea we're exploring is caching the generated source code for those query plans, and then compiling it as-needed later. If possible, we'd like to re-use the Calcite code responsible for this, though we'd need to relax the visibility modifier of EnumerableInterpretable#getBindable <https://github.com/apache/calcite/blob/b1308feff49c8b747b3bbb52e1519d334bc984ec/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpretable.java#L134> from package-private to public so it'd be invokable from our code. Do you have any thoughts on whether that'd be acceptable? Thank you, Austin On Tue, 1 Oct 2024 at 09:10, Stamatis Zampetakis <[email protected]> wrote: > Hey Austin, > > Indeed the most popular serialization/deserialization method for query > plans is by using the RelJsonReader and RelJsonWriter. However, we are > lacking a bit of test coverage around this feature so when new > RelNodes are added it's easy to miss the required changes. Moreover, > it's quite common for other projects to plug-in their own custom > RelNode's so serialization happens outside the classes present in > calcite core. We would happily accept contributions in this area so > feel free to create a JIRA ticket and submit a PR. > > Best, > Stamatis > > On Mon, Sep 30, 2024 at 11:55 AM Austin Richardson > <[email protected]> wrote: > > > > Hi Calcite devs, > > > > Our team is currently experimenting with ideas for serialising and > > deserialising optimised query plans using RelJsonWriter and > RelJsonReader. > > We noticed this is possible for pre-optimisation logical plans, but ran > > into issues when trying to do so for post-optimisation execution plans. > For > > example, LogicalJoin implements the required constructor for > serialisation > > here > > < > https://calcite.apache.org/javadocAggregate/org/apache/calcite/rel/logical/LogicalJoin.html#%3Cinit%3E(org.apache.calcite.rel.RelInput) > >, > > but the specific JOIN implementations (e.g. > EnumerableBatchNestedLoopJoin) > > do not. > > > > Considering this, is there a preferred way for serialising and > > deserialising optimised query plans? We wanted to ensure we aren’t > missing > > something here. > > > > Thank you, > > > > Austin >
