Hi Julian, Thanks for the quick response! Glad to know that I'm at least not mis-using the APIs. Investigate and cut a ticket if I find anything interesting.
Thanks, Logan On Wed, Jun 12, 2024, 4:59 AM Julian Hyde <[email protected]> wrote: > It’s possible thst you’re running into a performance bug - such as a loop > that is O[n ^ 2) in the number of SqlNodes - that has never been noticed > before. > > Can you do some investigation? Say, double the size or depth of the query > and see if the running time more than doubles. Or run the conversion > repeatedly, send sig-kill to your JVM to generate stack traces, and see > whether it is often inside the same piece of code. > > If those approaches yield results, log a bug. We would rather fix the bug, > and have people use SubqueryRemoveRule, rather than turning on isExpand. > > Julian > > > On Jun 11, 2024, at 7:28 PM, JinxTheKid <[email protected]> wrote: > > > > Hi Calcite community, > > > > I'm running into some challenges using SubqueryRemoveRule and was hoping > > the community could help me understand any gaps in my understanding. > > > > I am attempting to remove subqueries from row expressions in a relational > > plan. My motivation for doing this is that I want to call metadata > queries > > such GetColumnOrigins on my relational plan, but these tools don't seem > to > > work unless subqueries are removed. To remove subqueries, I am... > > - Using SqlToRelConverter to convert from SqlNode to RelRoot. > > - After obtaining the RelRoot, using a HepProgram with subquery remove > > rules to remove the subqueries. > > > > For small programs, this process works fine, however when testing against > > larger queries, ~2MB, the process of removing subqueries takes about as > > much time as converting a SqlNode to a RelRoot, effectively doubling the > > time to analyze a query. I've tried using a deprecated setting, > > SqlToRelConverter.Config.isExpand, which seems to provide results similar > > to what I want with minimal performance impact. > > > > Since SqlToRelConverter.Config.isExpand is deprecated, I assume that the > > non-deprecated way to expand subqueries would be as performant or better, > > but I am experiencing a worse performance. This leads me to believe that > > I'm using the APIs incorrectly, but I'm not sure what I am missing. > > > > Q: Is there a more efficient way in Calcite to remove subqueries than > what > > I've described above? Is there risk to using > > SqlToRelConverter.Config.isExpand even if it's deprecated? > > > > Q: Are there any other resources besides Java docs and GitHub projects I > > could refer to to learn more about my problem? > > > > Thanks for any help, > > Logan >
