Hi there!

In Pinot we want to work on a new optimization that lets us reuse some
parts of the query plan.
Basically what we want is to change our nodes to be able to send the same
data to multiple parent operators, transforming our trees into DAGs like
shown in this diagram from Vladimir Ozerov post in Querify Labs
<https://www.querifylabs.com/blog/data-shuffling-in-distributed-sql-engines>
:

[image: image.png]

I've looked for older messages in the dev mailing list and I found some
threads saying that the Calcite model is tree based and DAGs are not
supported. If that is the case I will have to implement this optimization
after the Calcite plan is generated, but I would like to avoid this because
we are trying to move more and more logic into Calcite procedures and this
would be a step back.

Reply via email to