Question about a possible bug in RelBuilder.aggregate_()
Hello Calcite contributors, I am working on a project that builds off of Calcite and I believe I may have encountered a bug in RelBuilder.aggregate_(). The problem that I have encountered is that this function is dropping traitSet information from my the Project node when making a copy. In particular, line 2479 calls project.copy() to prune any unused columns, but instead of reusing the traitSet from the original projection it takes the traitSet directly from the cluster. In my example, this caused problems for me because I was propagating convention information through the traitSet. I may be incorrectly relying on the traitSet in this case and perhaps my copy implementation should be improved, but I wanted to raise this as a possible concern in case anyone else has encountered this issue. I am happy to submit a PR if you agree this is a bug. Thanks for you time, Nick Riasanovsky
Re: Understanding the RelMdColumnUniqueness private constructor
Okay sounds good. Thank you Julian! On Mon, Aug 14, 2023 at 1:55 PM Julian Hyde wrote: > I checked a few other RelMdXxx classes, and. they have private > constructors too. Which makes sense, because you don’t extend metadata > providers by subclassing. You just define your own provider and add it to > the chain. > > RelMdColumnUniqueness has > > public static final RelMetadataProvider SOURCE = > ReflectiveRelMetadataProvider.reflectiveSource( > new RelMdColumnUniqueness(), > BuiltInMetadata.ColumnUniqueness.Handler.class); > > You could define a similar SOURCE in your class that chains > RelMdColumnUniqueness handlers with your own. > > Julian > > > > On Aug 14, 2023, at 9:32 AM, Nick Riasanovsky wrote: > > > > Hi everyone, > > > > I'm trying to understand why RelMdColumnUniqueness. I have interest in > > subclassing RelMdColumnUniqueness in my project and am unable to do so > > because the constructor is private. Other metadata queries don't appear > to > > have private constructors, such as RelMdRowCount. If anyone has any > > insights or context that would be very helpful. > > > > Thanks, > > Nick Riasanovsky > >
Understanding the RelMdColumnUniqueness private constructor
Hi everyone, I'm trying to understand why RelMdColumnUniqueness. I have interest in subclassing RelMdColumnUniqueness in my project and am unable to do so because the constructor is private. Other metadata queries don't appear to have private constructors, such as RelMdRowCount. If anyone has any insights or context that would be very helpful. Thanks, Nick Riasanovsky
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Hi Alessandro, The issue that I saw was that getPredicates() for a RelSubset has a section of code that returns an empty list if Bug.CALCITE_1048_FIXED isn't set to True. I thought this was my configuration issue because I am working on a project that isn't using the latest calcite release and I mixed up the issue with another, so I believed it was already fixed. After I took another today, I realized that this issue is still open so even the latest version of calcite won't be able to use this rule with RelSubset unless they switch Bug.CALCITE_1048_FIXED to True (which I presume is generally unsafe). Because of this, I'm wondering if there is any interest in updating AggregateProjectPullUpConstantsRule to have the input to `mq.getPulledUpPredicates()` depend on the status of this bug resolution. If the bug is fixed then it would work as currently written, but otherwise if aggregate.getInput() is a RelSubset we could use the input, which I believe should always be a subset of the information that could be available in the subset because it's a union. If this is any interest in this change I would be happy to submit a PR upstream with this behavior. Thanks again and sorry for the confusion, Nick Riasanovsky On Fri, Aug 4, 2023 at 3:52 AM Alessandro Solimando < alessandro.solima...@gmail.com> wrote: > Hey Nick, > thanks for confirming! > > If possible, it would be great to share a few more details, like what was > the misconfig and how you fixed it (while the symptoms are already > described in the first message). > > Saying this because our dev-list is also part of the project > documentation, your example can help others in the future. > > Best regards, > Alessandro > > On Fri, 4 Aug 2023 at 01:33, Nick Riasanovsky wrote: > > > Thank you for your response. There was an issue with my configuration > that > > I was able to resolve. > > > > On Thu, Aug 3, 2023 at 4:33 PM Julian Hyde > wrote: > > > > > The constraints on a RelSubset should be the union of the constraints > of > > > all of the RelNodes in that subset. (I haven’t tested it, or read the > > code. > > > But if it doesn’t do that, you’re probably seeing a bug, or you haven’r > > > configured your metadata providers correctly.) > > > > > > > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > > > > > > > Hello everyone, > > > > > > > > I am attempting to use AggregateProjectPullUpConstantsRule within an > > > > optimizer that is using the VolcanoPlanner. When doing so I encounter > > > > issues with what should be valid constants on the line > > > > `mq.getPulledUpPredicates()` because aggregate.getInput() is a > > RelSubset. > > > > As a result, the predicates list is always empty. If I replace the > uses > > > of > > > > aggregate.getInput() with input then everything works as expected. > > > > > > > > I am happy to submit a PR upstream with this fix, but I first want to > > > > confirm that my interpretation is correct as I am relatively new to > > using > > > > the volcano planner and could have missed a configuration step. > > > > > > > > Thanks everyone, > > > > Nick Riasanovsky > > > > > > > > >
Re: AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Thank you for your response. There was an issue with my configuration that I was able to resolve. On Thu, Aug 3, 2023 at 4:33 PM Julian Hyde wrote: > The constraints on a RelSubset should be the union of the constraints of > all of the RelNodes in that subset. (I haven’t tested it, or read the code. > But if it doesn’t do that, you’re probably seeing a bug, or you haven’r > configured your metadata providers correctly.) > > > On Aug 3, 2023, at 9:33 AM, Nick Riasanovsky wrote: > > > > Hello everyone, > > > > I am attempting to use AggregateProjectPullUpConstantsRule within an > > optimizer that is using the VolcanoPlanner. When doing so I encounter > > issues with what should be valid constants on the line > > `mq.getPulledUpPredicates()` because aggregate.getInput() is a RelSubset. > > As a result, the predicates list is always empty. If I replace the uses > of > > aggregate.getInput() with input then everything works as expected. > > > > I am happy to submit a PR upstream with this fix, but I first want to > > confirm that my interpretation is correct as I am relatively new to using > > the volcano planner and could have missed a configuration step. > > > > Thanks everyone, > > Nick Riasanovsky > >
AggregateProjectPullUpConstantsRule not working with the Volcano Planner
Hello everyone, I am attempting to use AggregateProjectPullUpConstantsRule within an optimizer that is using the VolcanoPlanner. When doing so I encounter issues with what should be valid constants on the line `mq.getPulledUpPredicates()` because aggregate.getInput() is a RelSubset. As a result, the predicates list is always empty. If I replace the uses of aggregate.getInput() with input then everything works as expected. I am happy to submit a PR upstream with this fix, but I first want to confirm that my interpretation is correct as I am relatively new to using the volcano planner and could have missed a configuration step. Thanks everyone, Nick Riasanovsky
[jira] [Created] (CALCITE-5892) SnowflakeSqlDialect doesn't provide custom unparseSqlIntervalLiteral support
Nick Riasanovsky created CALCITE-5892: - Summary: SnowflakeSqlDialect doesn't provide custom unparseSqlIntervalLiteral support Key: CALCITE-5892 URL: https://issues.apache.org/jira/browse/CALCITE-5892 Project: Calcite Issue Type: Bug Reporter: Nick Riasanovsky -- This message was sent by Atlassian Jira (v8.20.10#820010)
Using aliases later in a projection
Hello Calcite developers, I'm working on a project using Calcite and I'm looking to support certain SQL queries that are supported by Snowflake. Snowflake allows using references in the same SELECT statement in which they are created, such as: select 1 as X, X + 2 My understanding is that most SQL engines will not allow this so reasonably this is not supported by default in Calcite. However, I was wondering if there is any existing configuration that currently supports this or if anyone can give me some advice on how to support this. Thanks, Nick Riasanovsky
Changing the Parser
Hello calcite developers, I am working on a project that utilizes the calcite-core and I would like to extend the parser to support syntax that is custom to my application. I believe some of the additions I would like are supported by SQL offerings (such as <=> in SparkSQL) and could be included in the main project, but I have other additions that are unique to just my project. I found a stackoverflow post online that suggested I may need to clone calcite and build from source (I'm currently downloading the jar), but I am wondering if it is possible to extend the parser without maintaining my own clone. For context, I may need to add new TOKENs and I want to extend existing statements with additional input options, so I do not believe I can just add changes to the .fmpp file (although I would appreciate further clarification on how that works as well). Thanks, NIck Riasanovsky
Requesting a Review on a PR
I am a new contributor who opened a PR for an issue I opened on JIRA. Could I get some clarity on the process for getting someone to review my PR? I believe my PR should be ready. Thanks, Nick Riasanovsky
Joining JIRA as a contributor
Hi I'd like to join the JIRA contributor list. My Jira username is njriasan.
[jira] [Created] (CALCITE-4642) PlannerImpl ignores TypeSystems provided by FrameworkConfig
Nick Riasanovsky created CALCITE-4642: - Summary: PlannerImpl ignores TypeSystems provided by FrameworkConfig Key: CALCITE-4642 URL: https://issues.apache.org/jira/browse/CALCITE-4642 Project: Calcite Issue Type: Bug Affects Versions: 1.27.0 Reporter: Nick Riasanovsky Hi I'm relatively new to calcite, so my apologies if anything here isn't helpful. FrameworkConfig provides the options to set your own typeSystem via `Frameworks.newConfigBuilder().typeSystem(myTypeSystem)`. However, when later using this in a planner via `Frameworks.getPlanner(config)`, this new TypeSystem is always ignored in the implementation. Looking into the source code, it appears to me that there are two main issues. First, the PlannerImpl doesn't extract the typeSystem from the config [https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L108] and then when generating the typeFactory it always uses the default [https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java#L179.] I think this should be a relatively simple change and I'm happy to provide a PR, but since I'm new to calcite I will need some input on how to properly provide tests for this. I tested this issue with an example with a class that copies `RelDataTypeSystem.DEFAULT` except, for setting `shouldConvertRaggedUnionTypesToVarying`, but there is probably an easier way to test this. -- This message was sent by Atlassian Jira (v8.3.4#803005)