Serializabilty is not very popular in Java right now. There are a bunch of concerns, including security. Serializable classes are very brittle, because it's very easy to add a non-serializable field value in a sub-class.
I strongly favor externalizing over serialization. Convert RexNode to a serializable type (e.g. java.lang.String!), and then convert it back on the other side. Julian On Tue, Jul 7, 2020 at 7:25 PM Danny Chan <yuzhao....@gmail.com> wrote: > > Serialize the RexNode as Json format is a solution but I’m afraid it can not > solve the problem completely. > One problem with it is how to re-parse the json format back to RexNode, the > current RelJsonReader can only re-parse the RelNode but not RexNode, and it > needs the RelOptSchema to lookup the operators. > > In the distributed scenarios of Beam, I’m afraid it is hard to get the > RelOptSchema because it is execution, we usually see the RelOptSchema during > SQL compile time. > > Best, > Danny Chan > 在 2020年7月8日 +0800 AM3:39，Roman Kondakov <kondako...@mail.ru.invalid>，写道： > > Hi Rui, > > > > AFAIK, RelNodes can be serialized to and deserialized from JSON format. > > See test [1] as an example. If I understand it correct, RelNodes are > > serialized along with enclosed RexNodes, so you can transfer them over > > the network as plain strings. > > > > [1] > > https://github.com/apache/calcite/blob/f64cdcbb9f6535650f0227da19640e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > -- > > Roman Kondakov > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > Rui > > > > > > Il Mar 7 Lug 2020, 20:30 Rui Wang <amaliu...@apache.org> ha scritto: > > > > > > > Hi Community, > > > > > > > > In Apache Beam we are facing a use case where we need to keep RexNode in > > > > our distributed primitives. Because of the nature of distributed > > > > computing, > > > > Beam requires the usage of those primitives be serializable (thus those > > > > primitives can be sent over the network to backend/workers for > > > > further execution). > > > > > > > > In the Java world this requirement means to make RexNode implement the > > > > Java > > > > Serializable interface. > > > > > > > > A workaround right now is to create a bunch of classes to "clone" > > > > RexNode > > > > while making those classes implement the Serializable interface. > > > > > > > > > > Did you evaluate to use some framework like Kryo that allows you to > > > serialize Jon serializable classes? > > > > > > I think that in general Java serialisation is not efficient as it is too > > > general purpose. > > > It also brings in a few Security issues. > > > > > > Maybe an alternative idea is to add some serialisation ad-hoc mechanism in > > > RexNode. > > > We should also ensure that every RexNode will be able to be serialized and > > > deserialized. > > > > > > Enrico > > > > > > > > > > So what do you think of the idea that makes RexNode implement the > > > > Serializable interface? > > > > > > > > > > > > -Rui > > > > > > >
