[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Fix Version/s: 1.36.0 > Cannot fully disable trimming unused fields using Calcite configs > - > > Key: CALCITE-5984 > URL: https://issues.apache.org/jira/browse/CALCITE-5984 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.34.0 >Reporter: Wegdan Ghazi >Priority: Major > Labels: pull-request-available > Fix For: 1.36.0 > > > In Prepare#prepareSql there are 2 instances of trimming unused fields > 1. > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] > > {code:java} > // Trim unused fields. > root = trimUnusedFields(root); {code} > 2. > {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] > within > {code:java} > root = optimize(root, getMaterializations(), getLattices()); > {code} > I want to turn of the field trimming due to a personalisation I'm > implementing to scan a specific API, which I would think is possible using > [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] > I attempted to disable the trimming using hooks [PROGRAM, > SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming > program, yet on > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > {code:java} > // code placehprotected RelRoot trimUnusedFields(RelRoot root) { > final SqlToRelConverter.Config config = SqlToRelConverter.config() > .withTrimUnusedFields(shouldTrim(root.rel)) > .withExpand(THREAD_EXPAND.get()) > > .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); > final SqlToRelConverter converter = > getSqlToRelConverter(getSqlValidator(), catalogReader, config); > final boolean ordered = !root.collation.getFieldCollations().isEmpty(); > final boolean dml = SqlKind.DML.contains(root.kind); > return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); > } > {code} > The config passed (and edited by the hook) gets overwritten and only > _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to > happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. > Is this the intended behaviour? Any help is appreciated, thanks. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-5984: Labels: pull-request-available (was: ) > Cannot fully disable trimming unused fields using Calcite configs > - > > Key: CALCITE-5984 > URL: https://issues.apache.org/jira/browse/CALCITE-5984 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.34.0 >Reporter: Wegdan Ghazi >Priority: Major > Labels: pull-request-available > > In Prepare#prepareSql there are 2 instances of trimming unused fields > 1. > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] > > {code:java} > // Trim unused fields. > root = trimUnusedFields(root); {code} > 2. > {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] > within > {code:java} > root = optimize(root, getMaterializations(), getLattices()); > {code} > I want to turn of the field trimming due to a personalisation I'm > implementing to scan a specific API, which I would think is possible using > [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] > I attempted to disable the trimming using hooks [PROGRAM, > SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming > program, yet on > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > {code:java} > // code placehprotected RelRoot trimUnusedFields(RelRoot root) { > final SqlToRelConverter.Config config = SqlToRelConverter.config() > .withTrimUnusedFields(shouldTrim(root.rel)) > .withExpand(THREAD_EXPAND.get()) > > .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); > final SqlToRelConverter converter = > getSqlToRelConverter(getSqlValidator(), catalogReader, config); > final boolean ordered = !root.collation.getFieldCollations().isEmpty(); > final boolean dml = SqlKind.DML.contains(root.kind); > return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); > } > {code} > The config passed (and edited by the hook) gets overwritten and only > _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to > happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. > Is this the intended behaviour? Any help is appreciated, thanks. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-5984: - Fix Version/s: (was: 1.34.0) > Cannot fully disable trimming unused fields using Calcite configs > - > > Key: CALCITE-5984 > URL: https://issues.apache.org/jira/browse/CALCITE-5984 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.34.0 >Reporter: Wegdan Ghazi >Priority: Major > > In Prepare#prepareSql there are 2 instances of trimming unused fields > 1. > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] > > {code:java} > // Trim unused fields. > root = trimUnusedFields(root); {code} > 2. > {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] > within > {code:java} > root = optimize(root, getMaterializations(), getLattices()); > {code} > I want to turn of the field trimming due to a personalisation I'm > implementing to scan a specific API, which I would think is possible using > [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] > I attempted to disable the trimming using hooks [PROGRAM, > SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming > program, yet on > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > {code:java} > // code placehprotected RelRoot trimUnusedFields(RelRoot root) { > final SqlToRelConverter.Config config = SqlToRelConverter.config() > .withTrimUnusedFields(shouldTrim(root.rel)) > .withExpand(THREAD_EXPAND.get()) > > .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); > final SqlToRelConverter converter = > getSqlToRelConverter(getSqlValidator(), catalogReader, config); > final boolean ordered = !root.collation.getFieldCollations().isEmpty(); > final boolean dml = SqlKind.DML.contains(root.kind); > return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); > } > {code} > The config passed (and edited by the hook) gets overwritten and only > _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to > happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. > Is this the intended behaviour? Any help is appreciated, thanks. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Description: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); } {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. Is this the intended behaviour? Any help is appreciated, thanks. was: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); } {code} The config passed (and edited by the hook) gets overwritten and o
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Description: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); } {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. was: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Description: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. was: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [{_}PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER{_}], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable t
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Description: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [{_}PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER{_}], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. was: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [\{_}PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER\{_}], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/d
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Description: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [\{_}PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER\{_}], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by the hook) gets overwritten and only _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. Is this the intended behaviour? Any help is appreciated, thanks. was: In Prepare#prepareSql there are 2 instances of trimming unused fields 1. [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] {code:java} // Trim unused fields. root = trimUnusedFields(root); {code} 2. {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} called [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] within {code:java} root = optimize(root, getMaterializations(), getLattices()); {code} I want to turn of the field trimming due to a personalisation I'm implementing to scan a specific API, which I would think is possible using [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] I attempted to disable the trimming using hooks [{_}PROGRAM, SQL2REL_CONVERTER_CONFIG_BUILDER{_}], which succeeded to remove the trimming program, yet on [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] {code:java} // code placehprotected RelRoot trimUnusedFields(RelRoot root) { final SqlToRelConverter.Config config = SqlToRelConverter.config() .withTrimUnusedFields(shouldTrim(root.rel)) .withExpand(THREAD_EXPAND.get()) .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); final SqlToRelConverter converter = getSqlToRelConverter(getSqlValidator(), catalogReader, config); final boolean ordered = !root.collation.getFieldCollations().isEmpty(); final boolean dml = SqlKind.DML.contains(root.kind); return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); }older {code} The config passed (and edited by th
[jira] [Updated] (CALCITE-5984) Cannot fully disable trimming unused fields using Calcite configs
[ https://issues.apache.org/jira/browse/CALCITE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wegdan Ghazi updated CALCITE-5984: -- Environment: (was: Spring boot application integrating _org.apache.calcite_) > Cannot fully disable trimming unused fields using Calcite configs > - > > Key: CALCITE-5984 > URL: https://issues.apache.org/jira/browse/CALCITE-5984 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.34.0 >Reporter: Wegdan Ghazi >Priority: Major > Fix For: 1.34.0 > > > In Prepare#prepareSql there are 2 instances of trimming unused fields > 1. > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L297] > > {code:java} > // Trim unused fields. > root = trimUnusedFields(root); {code} > 2. > {color:#172b4d}[TrimFieldsProgram|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/tools/Programs.java#L369]{color} > called > [here|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L312] > within > {code:java} > root = optimize(root, getMaterializations(), getLattices()); > {code} > I want to turn of the field trimming due to a personalisation I'm > implementing to scan a specific API, which I would think is possible using > [SqlConverter.Config|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6085C20-L6085C26] > I attempted to disable the trimming using hooks [{_}PROGRAM, > SQL2REL_CONVERTER_CONFIG_BUILDER{_}], which succeeded to remove the trimming > program, yet on > [Prepare#trimUnusedFields|https://github.com/apache/calcite/blob/64268b9dd70bcdc15a3421ab120b8e5ecba17339/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L375] > {code:java} > // code placehprotected RelRoot trimUnusedFields(RelRoot root) { > final SqlToRelConverter.Config config = SqlToRelConverter.config() > .withTrimUnusedFields(shouldTrim(root.rel)) > .withExpand(THREAD_EXPAND.get()) > > .withInSubQueryThreshold(castNonNull(THREAD_INSUBQUERY_THRESHOLD.get())); > final SqlToRelConverter converter = > getSqlToRelConverter(getSqlValidator(), catalogReader, config); > final boolean ordered = !root.collation.getFieldCollations().isEmpty(); > final boolean dml = SqlKind.DML.contains(root.kind); > return root.withRel(converter.trimUnusedFields(dml || ordered, root.rel)); > }older > {code} > The config passed (and edited by the hook) gets overwritten and only > _shouldTrim(root.rel)_ can enable/disable trimming, I would expect this to > happen iff the config _withTrimUnusedFields_ is only set to {_}true{_}. > -- This message was sent by Atlassian Jira (v8.20.10#820010)