AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Hello everyone, I am attempting to use AggregateProjectPullUpConstantsRule within an optimizer that is using the VolcanoPlanner. When doing so I encounter issues with what should be valid constants on the line `mq.getPulledUpPredicates()` because aggregate.getInput() is a RelSubset. As a result, the predicates list is always empty. If I replace the uses of aggregate.getInput() with input then everything works as expected. I am happy to submit a PR upstream with this fix, but I first want to confirm that my interpretation is correct as I am relatively new to using the volcano planner and could have missed a configuration step. Thanks everyone, Nick Riasanovsky
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
The constraints on a RelSubset should be the union of the constraints of all of the RelNodes in that subset. (I haven’t tested it, or read the code. But if it doesn’t do that, you’re probably seeing a bug, or you haven’r configured your metadata providers correctly.) > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > Hello everyone, > > I am attempting to use AggregateProjectPullUpConstantsRule within an > optimizer that is using the VolcanoPlanner. When doing so I encounter > issues with what should be valid constants on the line > `mq.getPulledUpPredicates()` because aggregate.getInput() is a RelSubset. > As a result, the predicates list is always empty. If I replace the uses of > aggregate.getInput() with input then everything works as expected. > > I am happy to submit a PR upstream with this fix, but I first want to > confirm that my interpretation is correct as I am relatively new to using > the volcano planner and could have missed a configuration step. > > Thanks everyone, > Nick Riasanovsky
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Thank you for your response. There was an issue with my configuration that I was able to resolve. On Thu, Aug 3, 2023 at 4:33 PM Julian Hyde wrote: > The constraints on a RelSubset should be the union of the constraints of > all of the RelNodes in that subset. (I haven’t tested it, or read the code. > But if it doesn’t do that, you’re probably seeing a bug, or you haven’r > configured your metadata providers correctly.) > > > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > > > Hello everyone, > > > > I am attempting to use AggregateProjectPullUpConstantsRule within an > > optimizer that is using the VolcanoPlanner. When doing so I encounter > > issues with what should be valid constants on the line > > `mq.getPulledUpPredicates()` because aggregate.getInput() is a RelSubset. > > As a result, the predicates list is always empty. If I replace the uses > of > > aggregate.getInput() with input then everything works as expected. > > > > I am happy to submit a PR upstream with this fix, but I first want to > > confirm that my interpretation is correct as I am relatively new to using > > the volcano planner and could have missed a configuration step. > > > > Thanks everyone, > > Nick Riasanovsky > >
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Hey Nick, thanks for confirming! If possible, it would be great to share a few more details, like what was the misconfig and how you fixed it (while the symptoms are already described in the first message). Saying this because our dev-list is also part of the project documentation, your example can help others in the future. Best regards, Alessandro On Fri, 4 Aug 2023 at 01:33, Nick Riasanovsky wrote: > Thank you for your response. There was an issue with my configuration that > I was able to resolve. > > On Thu, Aug 3, 2023 at 4:33 PM Julian Hyde wrote: > > > The constraints on a RelSubset should be the union of the constraints of > > all of the RelNodes in that subset. (I haven’t tested it, or read the > code. > > But if it doesn’t do that, you’re probably seeing a bug, or you haven’r > > configured your metadata providers correctly.) > > > > > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > > > > > Hello everyone, > > > > > > I am attempting to use AggregateProjectPullUpConstantsRule within an > > > optimizer that is using the VolcanoPlanner. When doing so I encounter > > > issues with what should be valid constants on the line > > > `mq.getPulledUpPredicates()` because aggregate.getInput() is a > RelSubset. > > > As a result, the predicates list is always empty. If I replace the uses > > of > > > aggregate.getInput() with input then everything works as expected. > > > > > > I am happy to submit a PR upstream with this fix, but I first want to > > > confirm that my interpretation is correct as I am relatively new to > using > > > the volcano planner and could have missed a configuration step. > > > > > > Thanks everyone, > > > Nick Riasanovsky > > > > >
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Hi Alessandro, The issue that I saw was that getPredicates() for a RelSubset has a section of code that returns an empty list if Bug.CALCITE_1048_FIXED isn't set to True. I thought this was my configuration issue because I am working on a project that isn't using the latest calcite release and I mixed up the issue with another, so I believed it was already fixed. After I took another today, I realized that this issue is still open so even the latest version of calcite won't be able to use this rule with RelSubset unless they switch Bug.CALCITE_1048_FIXED to True (which I presume is generally unsafe). Because of this, I'm wondering if there is any interest in updating AggregateProjectPullUpConstantsRule to have the input to `mq.getPulledUpPredicates()` depend on the status of this bug resolution. If the bug is fixed then it would work as currently written, but otherwise if aggregate.getInput() is a RelSubset we could use the input, which I believe should always be a subset of the information that could be available in the subset because it's a union. If this is any interest in this change I would be happy to submit a PR upstream with this behavior. Thanks again and sorry for the confusion, Nick Riasanovsky On Fri, Aug 4, 2023 at 3:52 AM Alessandro Solimando < alessandro.solima...@gmail.com> wrote: > Hey Nick, > thanks for confirming! > > If possible, it would be great to share a few more details, like what was > the misconfig and how you fixed it (while the symptoms are already > described in the first message). > > Saying this because our dev-list is also part of the project > documentation, your example can help others in the future. > > Best regards, > Alessandro > > On Fri, 4 Aug 2023 at 01:33, Nick Riasanovsky wrote: > > > Thank you for your response. There was an issue with my configuration > that > > I was able to resolve. > > > > On Thu, Aug 3, 2023 at 4:33 PM Julian Hyde > wrote: > > > > > The constraints on a RelSubset should be the union of the constraints > of > > > all of the RelNodes in that subset. (I haven’t tested it, or read the > > code. > > > But if it doesn’t do that, you’re probably seeing a bug, or you haven’r > > > configured your metadata providers correctly.) > > > > > > > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > > > > > > > Hello everyone, > > > > > > > > I am attempting to use AggregateProjectPullUpConstantsRule within an > > > > optimizer that is using the VolcanoPlanner. When doing so I encounter > > > > issues with what should be valid constants on the line > > > > `mq.getPulledUpPredicates()` because aggregate.getInput() is a > > RelSubset. > > > > As a result, the predicates list is always empty. If I replace the > uses > > > of > > > > aggregate.getInput() with input then everything works as expected. > > > > > > > > I am happy to submit a PR upstream with this fix, but I first want to > > > > confirm that my interpretation is correct as I am relatively new to > > using > > > > the volcano planner and could have missed a configuration step. > > > > > > > > Thanks everyone, > > > > Nick Riasanovsky > > > > > > > > >
