Here's one (partial) example (truncated because it contains potential sensitive info, and didn't obfuscate or try to reproduce locally with non sensitive data): "rel#8643738:LogicalProject.NONE.ANY([]).[](input=rel#8643736:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643702:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643668:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643634:LogicalProject.NONE.ANY([]).[](input=rel#8643632:LogicalAggregate.NONE.ANY([]).[](input=rel#8643630:LogicalAggregate.NONE.ANY([]).[](input=rel#8643628:LogicalProject.NONE.ANY([]).[](input=rel#8643626:LogicalFilter.NONE.ANY([]).[](input=rel#8643624:LogicalProject.NONE.ANY([]).[](input=rel#8643622:LogicalProject.NONE.ANY([]).[](input=rel#8643842:MultiJoin.NONE.ANY([]).[](input#0=rel#8643838:LogicalProject.NONE.ANY([]).[](input=rel#8643615:MultiJoin.NONE.ANY([]).[](input#0=rel#8643603:LogicalProject.NONE.ANY([]).[](input=rel#8643601:SampleCrel.NONE.ANY([]).[](input=rel#8639853:ScanCrel.NONE.ANY([]).[](table="...
The Logical* relnodes don't override computeDigest method, so this is basically whatever AbstractRelNode#computeDigest is doing: https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/AbstractRelNode.java#L415 Laurent On Wed, Aug 15, 2018 at 1:57 PM Julian Hyde <jh...@apache.org> wrote: > I thought the digest only included the IDs of the inputs, not the digest > of the inputs. Am I mistaken? > > Could you give an example of large description & digest? > > > On Aug 15, 2018, at 1:46 PM, Laurent Goujon <laur...@dremio.com> wrote: > > > > Hi folks, > > > > I'm looking for some guidance here before opening JIRAs/pull requests. > > > > I'm examining a memory dump during a planning operation and a significant > > amount of memory are strings used for RelNode digest and description > (some > > strings being around 130kb). In that particular case, the relnode tree is > > particularly deep, and since the digest is basically done recursively, > the > > deepest/widest the tree, the longer the digest. > > > > The easy solution would be to not go deep when adding inputs to the > digest, > > and instead of adding the input description to only add their type, id > and > > traits (and also not recurse). Would this break parts of calcite, or > cause > > other inconvenience because some use-cases rely on digest/description to > be > > basically the whole tree in a textual form? > > > > Laurent > >
