[jira] [Resolved] (CALCITE-4882) Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval
[ https://issues.apache.org/jira/browse/CALCITE-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4882. - Resolution: Won't Fix Abandoning due to lack of interest/contention. A simpler fix in CALCITE-4952 satisfied the slowest case in Graal. > Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval > > > Key: CALCITE-4882 > URL: https://issues.apache.org/jira/browse/CALCITE-4882 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-4934) Deploy snapshot releases to Apache snapshot repo
[ https://issues.apache.org/jira/browse/CALCITE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4934. - Fix Version/s: 1.29.0 Assignee: Vladimir Sitnikov (was: Jacques Nadeau) Resolution: Fixed Was resolved when someone updated the Jenkins job. Don't believe there is a related commit. Jenkins job is here: https://ci-builds.apache.org/job/Calcite/job/Calcite-snapshots/ Estimated fix around version 1.29.0. > Deploy snapshot releases to Apache snapshot repo > > > Key: CALCITE-4934 > URL: https://issues.apache.org/jira/browse/CALCITE-4934 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Vladimir Sitnikov >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > It would be helpful to be able to test out snapshot builds of Calcite without > having to build locally. It seems like CALCITE-351 originally implemented > this but it hasn't been happening for many years. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-5008) Ignore synthetic and static methods in MetadataDef
[ https://issues.apache.org/jira/browse/CALCITE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-5008. - Resolution: Fixed Fixed in [cbbe5701b7f61d7f8df12d314ba5aabf898c1cae|https://github.com/apache/calcite/commit/cbbe5701b7f61d7f8df12d314ba5aabf898c1cae] > Ignore synthetic and static methods in MetadataDef > -- > > Key: CALCITE-5008 > URL: https://issues.apache.org/jira/browse/CALCITE-5008 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.29.0 >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: pull-request-available > Fix For: 1.30.0 > > Time Spent: 3h > Remaining Estimate: 0h > > According to [https://www.jacoco.org/jacoco/trunk/doc/faq.html] , Jacoco adds > a synthetic static method called $jacocoInit() during instrumentation. > MetadataDef constructor breaks if it encounters such method on the class > given to it because it does not expect it. But it looks like it simply should > not consider synthetic methods (as well as static methods) at all. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4991) Improve RuleEventLogger to also print input rels in FULL_PLAN mode
[ https://issues.apache.org/jira/browse/CALCITE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480134#comment-17480134 ] Jacques Nadeau commented on CALCITE-4991: - You already merged this but I was going to suggest that you actually base this on the set of things the rules operand observes. There are definitely cases the horizon is beyond the input. > Improve RuleEventLogger to also print input rels in FULL_PLAN mode > -- > > Key: CALCITE-4991 > URL: https://issues.apache.org/jira/browse/CALCITE-4991 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.29.0 >Reporter: Alessandro Solimando >Assignee: Alessandro Solimando >Priority: Major > Labels: pull-request-available > Fix For: 1.30.0 > > Time Spent: 20m > Remaining Estimate: 0h > > After using for some time now the new plan logging capabilities introduced > via _RuleEventLogger_ in CALCITE-4704, only printing the new rel produced by > a rule is not enough for complex cases, because the input rel(s) used by the > rule are not printed, and sometimes they are nowhere in the logs, making it > hard to understand what situation caused the appearance of a given plan of > interest. > The present ticket aims at additionally printing, for _FULL_PLAN_ marker, the > input rels used by a given rule, before printing the output rel produced by > the rule application. > The output logs would evolve from: > {noformat} > 022-01-21T02:41:27,466 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Apply rule > [HiveProjectFilterPullUpConstantsRule] to > [rel#45:HiveProject,rel#56:HiveFilter] > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Rule [HiveProjectFilterPullUpConstantsRule] > produced [rel#58:HiveProject] > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Full plan for [rel#58:HiveProject]: > HiveProject(month=[CAST(202110):INTEGER]) > HiveFilter(condition=[=($0, 202110)]) > HiveTableScan(table=[[default, test2]], table:alias=[test2]) > {noformat} > to: > {noformat} > 022-01-21T02:41:27,466 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Apply rule > [HiveProjectFilterPullUpConstantsRule] to > [rel#45:HiveProject,rel#56:HiveFilter] > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Full plan for rule input > [rel#45:HiveProject]: > HiveProject(month=[$0]) > HiveFilter(condition=[=($0, 202110)]) > HiveTableScan(table=[[default, test2]], table:alias=[test2]) > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Full plan for rule input [rel#56:HiveFilter]: > HiveFilter(condition=[=($0, 202110)]) > HiveTableScan(table=[[default, test2]], table:alias=[test2]) > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Rule [HiveProjectFilterPullUpConstantsRule] > produced [rel#58:HiveProject] > 2022-01-21T02:41:27,468 DEBUG [02c3a9eb-0565-404f-9c56-a2bbb556c827 main] > calcite.RuleEventLogger: call#5: Full plan for [rel#58:HiveProject]: > HiveProject(month=[CAST(202110):INTEGER]) > HiveFilter(condition=[=($0, 202110)]) > HiveTableScan(table=[[default, test2]], table:alias=[test2]) > {noformat} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4964) Reduce recursion when push predicate into CASE
[ https://issues.apache.org/jira/browse/CALCITE-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466649#comment-17466649 ] Jacques Nadeau commented on CALCITE-4964: - An additional note here: if we do adjust code for optimization reasons, I'd prefer for us to get into the habit of also implementing microbenchmarks for the code. Otherwise, future changes are likely to regress things. It also establishes a clear ask for future changes in this code where we can ask a contributor to report the benchmark impact before accepting additional changes. > Reduce recursion when push predicate into CASE > -- > > Key: CALCITE-4964 > URL: https://issues.apache.org/jira/browse/CALCITE-4964 > Project: Calcite > Issue Type: Improvement >Reporter: Ziwei Liu >Assignee: Ziwei Liu >Priority: Major > > I think CaseShuttle do not need a loop of for(;; ).When a predicate is pushed > into case, we only need to check the new RexCall whether need to push > predicate into the child operand, not need to do a new recursion. > So I think CaseShuttle can be reduced like: > {code:java} > @Override public RexNode visitCall(RexCall call) { > call = (RexCall) super.visitCall(call); > if (call instanceof OdpsLambdaRexCall) { > return call; > } > final RexCall old = call; > call = ReduceExpressionsRule.pushPredicateIntoCase(call); > if (call == old) { > return call; > } else { > boolean containCase = false; > List newOps = new ArrayList<>(call.getOperands().size()); > // call will be like case when c1 then f(x1 ...) > // check whether need push f into x1 > for (int i = 0; i < call.getOperands().size(); i ++) { > RexNode op = call.getOperands().get(i); > RexNode newOp = op; > if (i % 2 == 1 || i == call.getOperands().size() - 1) { > if (op instanceof RexCall) { > newOp = ReduceExpressionsRule.pushPredicateIntoCase((RexCall) > op); > } > } > if (op != newOp) { > containCase = true; > } > newOps.add(newOp); > } > if (!containCase) { > return call; > } else { > return call.clone(call.getType(), newOps); > } > } > } {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4952) Enable use of RelMetadataQuery using a simplistic & slow proxy path
[ https://issues.apache.org/jira/browse/CALCITE-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4952. - Fix Version/s: 1.30.0 Resolution: Fixed Resolved in [df35274e41d9a12e3b98010125e8248435673aa9|https://github.com/apache/calcite/commit/df35274e41d9a12e3b98010125e8248435673aa9] > Enable use of RelMetadataQuery using a simplistic & slow proxy path > --- > > Key: CALCITE-4952 > URL: https://issues.apache.org/jira/browse/CALCITE-4952 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.30.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > As part of working on CALCITE-4882 and CALCITE-4928, it became clear that > there should be a simple, non-janino implementation of RelMetadataQuery > internals to help with debugging. I'm not sure when/how but as of right now > the Reflective handlers can't be used with RelMetadataQuery without also > using Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4942) Deprecate boilerplate for Rel Metadata
[ https://issues.apache.org/jira/browse/CALCITE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17465853#comment-17465853 ] Jacques Nadeau commented on CALCITE-4942: - [~jamesstarr], I actually did several iterations on how to construct handlers to be cleaner/more modern/simpler. The best I came up with can be found in [this gist|https://gist.github.com/jacques-n/19e06a39704d789a06cdd06cffacc8c1]. There are two different things that I think are ideal to accomplish: - handlers are generic and declare the relnode interface they operate on. This creates an explicit functional definition of what metadata methods should look like (something that only exists implicitly today). - handlers have a generic method for metadata and use a generic definition. This allows a simplification of code that wants to invoke them since there are only 6 total functional signatures that exist across all metadata operations. (It's six to avoid boxing, otherwise it would be three.) If we're doing breaking changes, I think it would be helpful to introduce these patterns to not only clean up the old stuff but introducing a more standardized way to express these things. > Deprecate boilerplate for Rel Metadata > -- > > Key: CALCITE-4942 > URL: https://issues.apache.org/jira/browse/CALCITE-4942 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > To define a Rel metadata type you need to create 2 class, 2 methods and a > static variable. However, current implementation of RelMetadata back by > Janino only require the Handler class a single method signature. The current > metadata handler interface also has a generic that provides no type safety > through out code. > Currently to define a rel metadata type: > {code:java} > public class MyMetadata extends Metadata { > MetadataDef DEF = ... > > VALUE myMethod1(); > class Handler extends RelHandler { > VALUE myMethod2(RelNode, MetadataQuery) > } > } > {code} > What is actually needed to define a rel metadata type: > {code:java} > class MyMetadataHandler extends RelHandler { > VALUE myMethod(RelNode, MetadataQuery) > } > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4952) Enable use of RelMetadataQuery using a simplistic & slow proxy path
[ https://issues.apache.org/jira/browse/CALCITE-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462274#comment-17462274 ] Jacques Nadeau edited comment on CALCITE-4952 at 12/19/21, 7:11 PM: Thanks [~zabetak]. I have used that before. I probably understated the goals here. Some additional goals to this ticket: - Ensure that the metadata apis can work with multiple implementations, not just janino. We have some issues in this code where a bunch of things became highly coupled to Janino. This makes building alternatives against standard apis like RelMetadataProvider very difficult (definitely worse before CALCITE-4928 was merged). By having a second reflection/proxy simplistic implementation, we validate that the apis are healthy/reasonable extension points. - I continue to target GraalVM native where you can't use Janino at all. Having even a basic, poorly-performing alternative is better than nothing. I ultimately plan to get CALCITE-4882 merged to provide a high performance Janino alternative but that is a much bigger patch. Just having \*any\* solution would be a big improvement right now. was (Author: jnadeau): Thanks [~zabetak]. I have used that before. I probably understated the goals here. Some additional goals to this ticket: - Ensure that the metadata apis can work with multiple implementations, not just janino. We have some issues in this code where a bunch of things became highly coupled to Janino. This makes building alternatives against standard apis like RelMetadataProvider be very difficult to actually use (was the case until CALCITE-4928 was merged). By having a second reflection/proxy simplistic implementation, we validate that the apis are healthy/reasonable extension points. - I continue to target GraalVM native where you can't use Janino at all. Having even a basic, poorly-performing alternative is better than nothing. I ultimately plan to get CALCITE-4882 merged to provide a high performance Janino alternative but that is a much bigger patch. Just having \*any\* solution would be a big improvement right now. > Enable use of RelMetadataQuery using a simplistic & slow proxy path > --- > > Key: CALCITE-4952 > URL: https://issues.apache.org/jira/browse/CALCITE-4952 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of working on CALCITE-4882 and CALCITE-4928, it became clear that > there should be a simple, non-janino implementation of RelMetadataQuery > internals to help with debugging. I'm not sure when/how but as of right now > the Reflective handlers can't be used with RelMetadataQuery without also > using Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4952) Enable use of RelMetadataQuery using a simplistic & slow proxy path
[ https://issues.apache.org/jira/browse/CALCITE-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462274#comment-17462274 ] Jacques Nadeau commented on CALCITE-4952: - Thanks [~zabetak]. I have used that before. I probably understated the goals here. Some additional goals to this ticket: - Ensure that the metadata apis can work with multiple implementations, not just janino. We have some issues in this code where a bunch of things became highly coupled to Janino. This makes building alternatives against standard apis like RelMetadataProvider be very difficult to actually use (was the case until CALCITE-4928 was merged). By having a second reflection/proxy simplistic implementation, we validate that the apis are healthy/reasonable extension points. - I continue to target GraalVM native where you can't use Janino at all. Having even a basic, poorly-performing alternative is better than nothing. I ultimately plan to get CALCITE-4882 merged to provide a high performance Janino alternative but that is a much bigger patch. Just having \*any\* solution would be a big improvement right now. > Enable use of RelMetadataQuery using a simplistic & slow proxy path > --- > > Key: CALCITE-4952 > URL: https://issues.apache.org/jira/browse/CALCITE-4952 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of working on CALCITE-4882 and CALCITE-4928, it became clear that > there should be a simple, non-janino implementation of RelMetadataQuery > internals to help with debugging. I'm not sure when/how but as of right now > the Reflective handlers can't be used with RelMetadataQuery without also > using Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4952) Enable use of RelMetadataQuery using a simplistic & slow proxy path
Jacques Nadeau created CALCITE-4952: --- Summary: Enable use of RelMetadataQuery using a simplistic & slow proxy path Key: CALCITE-4952 URL: https://issues.apache.org/jira/browse/CALCITE-4952 Project: Calcite Issue Type: Improvement Components: core Reporter: Jacques Nadeau Assignee: Jacques Nadeau As part of working on CALCITE-4882 and CALCITE-4928, it became clear that there should be a simple, non-janino implementation of RelMetadataQuery internals to help with debugging. I'm not sure when/how but as of right now the Reflective handlers can't be used with RelMetadataQuery without also using Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license
[ https://issues.apache.org/jira/browse/CALCITE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4948. - Resolution: Fixed Resolved in [e4cdceec1|https://github.com/apache/calcite/commit/e4cdceec1626da113367ce6d730e504509aa0e67] > Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license > - > > Key: CALCITE-4948 > URL: https://issues.apache.org/jira/browse/CALCITE-4948 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4949) Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless dependency
[ https://issues.apache.org/jira/browse/CALCITE-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4949: Description: There was an old comment [here|https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] that said: {code} Elastic search doesn't export painless script artifact to maven central. Using 3rd party version (codelibs) This JAR is used only in tests {code} This was referring to org.codelibs.elasticsearch.module:lang-painless dependency. Is this still true or can we update this to a proper first-party jar? Came up during review of CALCITE-4948 was: There was an old comment [here|https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] that said: {code} Elastic search doesn't export painless script artifact to maven central. Using 3rd party version (codelibs) This JAR is used only in tests {code} Is this still true or should be updated to a proper first-party jar? Came up during review of CALCITE-4948 > Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless > dependency > > > Key: CALCITE-4949 > URL: https://issues.apache.org/jira/browse/CALCITE-4949 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > > There was an old comment > [here|https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] > that said: > {code} > Elastic search doesn't export painless script artifact to maven central. > Using 3rd party version (codelibs) > This JAR is used only in tests > {code} > This was referring to org.codelibs.elasticsearch.module:lang-painless > dependency. Is this still true or can we update this to a proper first-party > jar? > Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4949) Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless dependency
[ https://issues.apache.org/jira/browse/CALCITE-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4949: Description: There was an old comment [here|https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] that said: {code} Elastic search doesn't export painless script artifact to maven central. Using 3rd party version (codelibs) This JAR is used only in tests {code} Is this still true or should be updated to a proper first-party jar? Came up during review of CALCITE-4948 was: There was an old comment [here|(https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] that said: {code} Elastic search doesn't export painless script artifact to maven central. Using 3rd party version (codelibs) This JAR is used only in tests {code} Is this still true or should be updated to a proper first-party jar? Came up during review of CALCITE-4948 > Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless > dependency > > > Key: CALCITE-4949 > URL: https://issues.apache.org/jira/browse/CALCITE-4949 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > > There was an old comment > [here|https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] > that said: > {code} > Elastic search doesn't export painless script artifact to maven central. > Using 3rd party version (codelibs) > This JAR is used only in tests > {code} > Is this still true or should be updated to a proper first-party jar? > Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (CALCITE-4949) Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate dependency
[ https://issues.apache.org/jira/browse/CALCITE-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4949: --- Assignee: ZheHu > Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate > dependency > > > Key: CALCITE-4949 > URL: https://issues.apache.org/jira/browse/CALCITE-4949 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > > There was an old comment > [here|(https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] > that said: > {code} > Elastic search doesn't export painless script artifact to maven central. > Using 3rd party version (codelibs) > This JAR is used only in tests > {code} > Is this still true or should be updated to a proper first-party jar? > Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4949) Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless dependency
[ https://issues.apache.org/jira/browse/CALCITE-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4949: Summary: Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless dependency (was: Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate dependency) > Evaluate if using "org.codelibs" is appropriate for an Elasticsearch painless > dependency > > > Key: CALCITE-4949 > URL: https://issues.apache.org/jira/browse/CALCITE-4949 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > > There was an old comment > [here|(https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] > that said: > {code} > Elastic search doesn't export painless script artifact to maven central. > Using 3rd party version (codelibs) > This JAR is used only in tests > {code} > Is this still true or should be updated to a proper first-party jar? > Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4949) Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate dependency
[ https://issues.apache.org/jira/browse/CALCITE-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461661#comment-17461661 ] Jacques Nadeau commented on CALCITE-4949: - [~VAE], can you take a look at this? > Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate > dependency > > > Key: CALCITE-4949 > URL: https://issues.apache.org/jira/browse/CALCITE-4949 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > > There was an old comment > [here|(https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] > that said: > {code} > Elastic search doesn't export painless script artifact to maven central. > Using 3rd party version (codelibs) > This JAR is used only in tests > {code} > Is this still true or should be updated to a proper first-party jar? > Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license
[ https://issues.apache.org/jira/browse/CALCITE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4948: --- Assignee: Jacques Nadeau (was: ZheHu) > Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license > - > > Key: CALCITE-4948 > URL: https://issues.apache.org/jira/browse/CALCITE-4948 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license
[ https://issues.apache.org/jira/browse/CALCITE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4948: --- Assignee: ZheHu > Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license > - > > Key: CALCITE-4948 > URL: https://issues.apache.org/jira/browse/CALCITE-4948 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Assignee: ZheHu >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license
[ https://issues.apache.org/jira/browse/CALCITE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4948: Component/s: elasticsearch-adapter > Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license > - > > Key: CALCITE-4948 > URL: https://issues.apache.org/jira/browse/CALCITE-4948 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4949) Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate dependency
Jacques Nadeau created CALCITE-4949: --- Summary: Evaluate if org.codelibs.elasticsearch.module:lang-painless is an appropriate dependency Key: CALCITE-4949 URL: https://issues.apache.org/jira/browse/CALCITE-4949 Project: Calcite Issue Type: Improvement Components: elasticsearch-adapter Reporter: Jacques Nadeau There was an old comment [here|(https://github.com/apache/calcite/blob/94e6272d57478a5913cb51e7df5c25e67511229a/elasticsearch/pom.xml] that said: {code} Elastic search doesn't export painless script artifact to maven central. Using 3rd party version (codelibs) This JAR is used only in tests {code} Is this still true or should be updated to a proper first-party jar? Came up during review of CALCITE-4948 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license
[ https://issues.apache.org/jira/browse/CALCITE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4948: Summary: Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license (was: Revert Elasticsearch to 7.10.2 to avoid extra repo and category x license) > Revert Elasticsearch to 7.10.2 to avoid extra repo and category X license > - > > Key: CALCITE-4948 > URL: https://issues.apache.org/jira/browse/CALCITE-4948 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Fix For: 1.29.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4948) Revert Elasticsearch to 7.10.2 to avoid extra repo and category x license
Jacques Nadeau created CALCITE-4948: --- Summary: Revert Elasticsearch to 7.10.2 to avoid extra repo and category x license Key: CALCITE-4948 URL: https://issues.apache.org/jira/browse/CALCITE-4948 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Fix For: 1.29.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4942) Deprecate Bolier plate for Rel Metadata
[ https://issues.apache.org/jira/browse/CALCITE-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460919#comment-17460919 ] Jacques Nadeau commented on CALCITE-4942: - Glad you're picking this up. Will be interested how you propose doing this without having breaking changes (and refactoring reflection reliance on existing getDef() functionality) > Deprecate Bolier plate for Rel Metadata > --- > > Key: CALCITE-4942 > URL: https://issues.apache.org/jira/browse/CALCITE-4942 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > > To define a Rel metadata type you need to create 2 class, 2 methods and a > static variable. However, current implementation of RelMetadata back by > Janino only require the Handler class a single method signature. The current > metadata handler interface also has a generic that provides no type safety > through out code. > Currently to define a rel metadata type: > {code:java} > public class MyMetadata extends Metadata { > MetadataDef DEF = ... > > VALUE myMethod1(); > class Handler extends RelHandler { > VALUE myMethod2(RelNode, MetadataQuery) > } > } > {code} > What is actually needed to define a rel metadata type: > {code:java} > class MyMetadataHandler extends RelHandler { > VALUE myMethod(RelNode, MetadataQuery) > } > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4898) Upgrading Elasticsearch version from 7.0.1 to 7.15 in Elasticsearch Adapter
[ https://issues.apache.org/jira/browse/CALCITE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4898: Fix Version/s: 1.29.0 > Upgrading Elasticsearch version from 7.0.1 to 7.15 in Elasticsearch Adapter > --- > > Key: CALCITE-4898 > URL: https://issues.apache.org/jira/browse/CALCITE-4898 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Affects Versions: 1.28.0 >Reporter: ZheHu >Assignee: ZheHu >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The current Elasticsearch version in Calcite is 7.0.1, while the latest > released ES version is 7.15.2. > There are many new SQL features added and bugs fixed from 7.0.1 to 7.15. By > upgrading the embedded ES, we can make good use of ES's new features, > especially about DSL, which enable developers to find one more suitable or > effective way to implement features and improve existing functions. > Since low level rest client has good compatibility among 7.x versions, > RestClient is also upgraded to 7.15.2, which is supported currently. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4898) Upgrading Elasticsearch version from 7.0.1 to 7.15 in Elasticsearch Adapter
[ https://issues.apache.org/jira/browse/CALCITE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4898. - Resolution: Fixed Merged in [d6a36fc08715eba31d1d6bc26ef1ff5558d04b8d|https://github.com/apache/calcite/commit/d6a36fc08715eba31d1d6bc26ef1ff5558d04b8d] > Upgrading Elasticsearch version from 7.0.1 to 7.15 in Elasticsearch Adapter > --- > > Key: CALCITE-4898 > URL: https://issues.apache.org/jira/browse/CALCITE-4898 > Project: Calcite > Issue Type: Improvement > Components: elasticsearch-adapter >Affects Versions: 1.28.0 >Reporter: ZheHu >Assignee: ZheHu >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > The current Elasticsearch version in Calcite is 7.0.1, while the latest > released ES version is 7.15.2. > There are many new SQL features added and bugs fixed from 7.0.1 to 7.15. By > upgrading the embedded ES, we can make good use of ES's new features, > especially about DSL, which enable developers to find one more suitable or > effective way to implement features and improve existing functions. > Since low level rest client has good compatibility among 7.x versions, > RestClient is also upgraded to 7.15.2, which is supported currently. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4929) Add default methods for getDef on metadata handlers
[ https://issues.apache.org/jira/browse/CALCITE-4929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4929. - Fix Version/s: 1.29.0 Resolution: Fixed > Add default methods for getDef on metadata handlers > --- > > Key: CALCITE-4929 > URL: https://issues.apache.org/jira/browse/CALCITE-4929 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Right now, metadata handler implementations must implement a getDef() method. > The values are singletons next to the interface definitions. This change > proposes making these singletons be returned automatically by default methods > rather than implementers needing to implement these known values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4929) Add default methods for getDef on metadata handlers
[ https://issues.apache.org/jira/browse/CALCITE-4929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458824#comment-17458824 ] Jacques Nadeau edited comment on CALCITE-4929 at 12/14/21, 12:24 AM: - Merged in [4ff5fa89c180e|https://github.com/apache/calcite/commit/4ff5fa89c180ebc30d8fb324c2a50b0e9797b9ca] Will resolve once editing perms comes back. was (Author: jacques): Merged in [4ff5fa89c180e|[https://github.com/apache/calcite/commit/4ff5fa89c180ebc30d8fb324c2a50b0e9797b9ca]] Will resolve once editing perms comes back. > Add default methods for getDef on metadata handlers > --- > > Key: CALCITE-4929 > URL: https://issues.apache.org/jira/browse/CALCITE-4929 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Right now, metadata handler implementations must implement a getDef() method. > The values are singletons next to the interface definitions. This change > proposes making these singletons be returned automatically by default methods > rather than implementers needing to implement these known values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4929) Add default methods for getDef on metadata handlers
[ https://issues.apache.org/jira/browse/CALCITE-4929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458824#comment-17458824 ] Jacques Nadeau edited comment on CALCITE-4929 at 12/14/21, 12:24 AM: - Merged in [4ff5fa89c180e|[https://github.com/apache/calcite/commit/4ff5fa89c180ebc30d8fb324c2a50b0e9797b9ca]] Will resolve once editing perms comes back. was (Author: jacques): Merged in [4ff5fa89c180e|[https://github.com/apache/calcite/commit/4ff5fa89c180ebc30d8fb324c2a50b0e9797b9ca].] Will resolve once editing perms comes back. > Add default methods for getDef on metadata handlers > --- > > Key: CALCITE-4929 > URL: https://issues.apache.org/jira/browse/CALCITE-4929 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Right now, metadata handler implementations must implement a getDef() method. > The values are singletons next to the interface definitions. This change > proposes making these singletons be returned automatically by default methods > rather than implementers needing to implement these known values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4929) Add default methods for getDef on metadata handlers
[ https://issues.apache.org/jira/browse/CALCITE-4929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458824#comment-17458824 ] Jacques Nadeau commented on CALCITE-4929: - Merged in [4ff5fa89c180e|[https://github.com/apache/calcite/commit/4ff5fa89c180ebc30d8fb324c2a50b0e9797b9ca].] Will resolve once editing perms comes back. > Add default methods for getDef on metadata handlers > --- > > Key: CALCITE-4929 > URL: https://issues.apache.org/jira/browse/CALCITE-4929 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Right now, metadata handler implementations must implement a getDef() method. > The values are singletons next to the interface definitions. This change > proposes making these singletons be returned automatically by default methods > rather than implementers needing to implement these known values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4934) Deploy snapshot releases to Apache snapshot repo
[ https://issues.apache.org/jira/browse/CALCITE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458553#comment-17458553 ] Jacques Nadeau commented on CALCITE-4934: - I've confirmed that this is working after INFRA-22601 was completed. > Deploy snapshot releases to Apache snapshot repo > > > Key: CALCITE-4934 > URL: https://issues.apache.org/jira/browse/CALCITE-4934 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > It would be helpful to be able to test out snapshot builds of Calcite without > having to build locally. It seems like CALCITE-351 originally implemented > this but it hasn't been happening for many years. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458094#comment-17458094 ] Jacques Nadeau edited comment on CALCITE-4935 at 12/12/21, 11:52 PM: - I just tried it with RC0. Yes, it logs (I didn't try to configure things). See example below. Note that I think [~zabetak] comments above were when he was trying to shade things, not the original rc0. {code:java} % java -jar avatica-standalone-server-1.20.0-shadow.jar -u jdbc:calcite WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. Not configuring Avatica authentication 2021-12-12T15:50:45,568 [main] INFO metrics.MetricsSystemLoader - No metrics implementation available on classpath. Using No-op implementation 2021-12-12T15:50:45,582 [main] INFO util.log - Logging initialized @451ms to org.eclipse.jetty.util.log.Slf4jLog 2021-12-12T15:50:45,752 [main] INFO server.Server - jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 17+35-LTS 2021-12-12T15:50:45,810 [main] INFO server.AbstractConnector - Started ServerConnector@2e6a5539{HTTP/1.1, (http/1.1)}{0.0.0.0:52894} 2021-12-12T15:50:45,810 [main] INFO server.Server - Started @680ms 2021-12-12T15:50:45,812 [main] INFO server.HttpServer - Service listening on port 52894. 2021-12-12T15:50:45,812 [main] INFO standalone.StandaloneServer - Started Avatica server on port 52894 with serialization PROTOBUF {code} was (Author: jacques): I just tried it with RC0. Yes, it logs. See example below. Note that I think [~zabetak] comments above were when he was trying to shade things, not the original rc0. {code:java} % java -jar avatica-standalone-server-1.20.0-shadow.jar -u jdbc:calcite WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. Not configuring Avatica authentication 2021-12-12T15:50:45,568 [main] INFO metrics.MetricsSystemLoader - No metrics implementation available on classpath. Using No-op implementation 2021-12-12T15:50:45,582 [main] INFO util.log - Logging initialized @451ms to org.eclipse.jetty.util.log.Slf4jLog 2021-12-12T15:50:45,752 [main] INFO server.Server - jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 17+35-LTS 2021-12-12T15:50:45,810 [main] INFO server.AbstractConnector - Started ServerConnector@2e6a5539{HTTP/1.1, (http/1.1)}{0.0.0.0:52894} 2021-12-12T15:50:45,810 [main] INFO server.Server - Started @680ms 2021-12-12T15:50:45,812 [main] INFO server.HttpServer - Service listening on port 52894. 2021-12-12T15:50:45,812 [main] INFO standalone.StandaloneServer - Started Avatica server on port 52894 with serialization PROTOBUF {code} > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458094#comment-17458094 ] Jacques Nadeau commented on CALCITE-4935: - I just tried it with RC0. Yes, it logs. See example below. Note that I think [~zabetak] comments above were when he was trying to shade things, not the original rc0. {code:java} % java -jar avatica-standalone-server-1.20.0-shadow.jar -u jdbc:calcite WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. Not configuring Avatica authentication 2021-12-12T15:50:45,568 [main] INFO metrics.MetricsSystemLoader - No metrics implementation available on classpath. Using No-op implementation 2021-12-12T15:50:45,582 [main] INFO util.log - Logging initialized @451ms to org.eclipse.jetty.util.log.Slf4jLog 2021-12-12T15:50:45,752 [main] INFO server.Server - jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 17+35-LTS 2021-12-12T15:50:45,810 [main] INFO server.AbstractConnector - Started ServerConnector@2e6a5539{HTTP/1.1, (http/1.1)}{0.0.0.0:52894} 2021-12-12T15:50:45,810 [main] INFO server.Server - Started @680ms 2021-12-12T15:50:45,812 [main] INFO server.HttpServer - Service listening on port 52894. 2021-12-12T15:50:45,812 [main] INFO standalone.StandaloneServer - Started Avatica server on port 52894 with serialization PROTOBUF {code} > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457809#comment-17457809 ] Jacques Nadeau edited comment on CALCITE-4935 at 12/12/21, 4:34 AM: Given [~elserj]'s comments here, it doesn't seem like the target for the standalone server is as a dependency. If that is the case, I think we should close this as won't fix and instead simply add some release notes commenting on the change. As I mentioned above, if a person is using this as a standalone daemon, it would best if we make it as easy (standard) as possible to configure logging. was (Author: jnadeau): Given [~elserj]'s comments here, it doesn't seem like the target for the standalone server is as a dependency. If that is the case, I think we should close this as won't fix and instead simply add some release notes commenting on the change. As I mentioned above, if a person is using this as a standalone daemon, we would make it as easy as possible to configure logging, etc. > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457809#comment-17457809 ] Jacques Nadeau commented on CALCITE-4935: - Given [~elserj]'s comments here, it doesn't seem like the target for the standalone server is as a dependency. If that is the case, I think we should close this as won't fix and instead simply add some release notes commenting on the change. As I mentioned above, if a person is using this as a standalone daemon, we would make it as easy as possible to configure logging, etc. > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457780#comment-17457780 ] Jacques Nadeau commented on CALCITE-4935: - Hey [~zabetak], You probably need to enable several resource transformers. For maven's shade plugin they are here: https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html. For example, the meta-inf example needs ServicesResourceTransformer. For gradle shadowjar, I'm not sure the similar item. It looks like maybe "mergeServiceFiles()" per https://github.com/johnrengelman/shadow/issues/105 > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1745#comment-1745 ] Jacques Nadeau edited comment on CALCITE-4935 at 12/12/21, 12:09 AM: - I don't really understand the expectation of shading here. Generally speaking, things should be one of the two following options: 1. This is a jar used within another application (far dependency jar). In that case, the standard pattern is to only include slf4j apis (unshaded) and let users bring their own logging implementation. 2. This is a jar used as a standalone server (easy to run/deploy single jar). In that case, there is no reason to shade dependencies as it isn't designed to be run in another application. People should be able to use standard logging configuration patterns. Including logging impl but shading it makes it hard to deal with in #1 and difficult to configure/control logging consistently in #2. That being said, if we really think the impl should be included and shaded (neither 1 nor 2 above), it looks like there may be need to merge some internal log4j2 files using a special transformer. See https://github.com/johnrengelman/shadow/issues/207, https://github.com/edwgiz/maven-shaded-log4j-transformer and https://issues.apache.org/jira/browse/LOG4J2-673. was (Author: jnadeau): I don't really understand the expectation of shading here. Generally speaking, things should be one of the two following options: 1. This is a jar used within another application (far dependency jar). In that case, the standard pattern is to only include slf4j apis (unshaded) and let users bring their own logging implementation. 2. This is a jar used as a standalone server (easy to run/deploy single jar). In that case, there is no reason to shade dependencies as it isn't designed to be run in another application. People should be able to use standard logging configuration patterns. Including logging impl but shading it makes it hard to deal configure/control logging consistently in #1. That being said, if we really think the impl should be included and shaded (neither 1 nor 2 above), it looks like there may be need to merge some internal log4j2 files using a special transformer. See https://github.com/johnrengelman/shadow/issues/207, https://github.com/edwgiz/maven-shaded-log4j-transformer and https://issues.apache.org/jira/browse/LOG4J2-673. > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1745#comment-1745 ] Jacques Nadeau commented on CALCITE-4935: - I don't really understand the expectation of shading here. Generally speaking, things should be one of the two following options: 1. This is a jar used within another application (far dependency jar). In that case, the standard pattern is to only include slf4j apis (unshaded) and let users bring their own logging implementation. 2. This is a jar used as a standalone server (easy to run/deploy single jar). In that case, there is no reason to shade dependencies as it isn't designed to be run in another application. People should be able to use standard patterns to configure. Including logging impl but shading it makes it hard to deal configure/control logging consistently in #1. That being said, if we really think the impl should be included and shaded (neither 1 nor 2 above), it looks like there may be need to merge some internal log4j2 files using a special transformer. See https://github.com/johnrengelman/shadow/issues/207, https://github.com/edwgiz/maven-shaded-log4j-transformer and https://issues.apache.org/jira/browse/LOG4J2-673. > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1745#comment-1745 ] Jacques Nadeau edited comment on CALCITE-4935 at 12/12/21, 12:07 AM: - I don't really understand the expectation of shading here. Generally speaking, things should be one of the two following options: 1. This is a jar used within another application (far dependency jar). In that case, the standard pattern is to only include slf4j apis (unshaded) and let users bring their own logging implementation. 2. This is a jar used as a standalone server (easy to run/deploy single jar). In that case, there is no reason to shade dependencies as it isn't designed to be run in another application. People should be able to use standard logging configuration patterns. Including logging impl but shading it makes it hard to deal configure/control logging consistently in #1. That being said, if we really think the impl should be included and shaded (neither 1 nor 2 above), it looks like there may be need to merge some internal log4j2 files using a special transformer. See https://github.com/johnrengelman/shadow/issues/207, https://github.com/edwgiz/maven-shaded-log4j-transformer and https://issues.apache.org/jira/browse/LOG4J2-673. was (Author: jnadeau): I don't really understand the expectation of shading here. Generally speaking, things should be one of the two following options: 1. This is a jar used within another application (far dependency jar). In that case, the standard pattern is to only include slf4j apis (unshaded) and let users bring their own logging implementation. 2. This is a jar used as a standalone server (easy to run/deploy single jar). In that case, there is no reason to shade dependencies as it isn't designed to be run in another application. People should be able to use standard patterns to configure. Including logging impl but shading it makes it hard to deal configure/control logging consistently in #1. That being said, if we really think the impl should be included and shaded (neither 1 nor 2 above), it looks like there may be need to merge some internal log4j2 files using a special transformer. See https://github.com/johnrengelman/shadow/issues/207, https://github.com/edwgiz/maven-shaded-log4j-transformer and https://issues.apache.org/jira/browse/LOG4J2-673. > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4935: Attachment: screenshot-1.png > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4935) Avatica standalone-server shades log4j without relocation
[ https://issues.apache.org/jira/browse/CALCITE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457757#comment-17457757 ] Jacques Nadeau commented on CALCITE-4935: - [~elserj], I see in the CALCITE-4152 that you removed three lines from the shadowjar implementation. Was that on purpose? Any idea why you did that? !screenshot-1.png! > Avatica standalone-server shades log4j without relocation > - > > Key: CALCITE-4935 > URL: https://issues.apache.org/jira/browse/CALCITE-4935 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: avatica-1.20.0 > > Attachments: screenshot-1.png > > > The issue has been found during the vote for avatica-1.20.0 RC0. > The standalone-server jar in the [staged maven > repository|https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/avatica/avatica-standalone-server/1.20.0/avatica-standalone-server-1.20.0.jar] > contains log4j2 classes but they are not relocated. > In previous, Avatica versions (e.g., avatica-1.19.0), log4j classes were all > relocated to avoid classpath problems with other libraries/apps potentially > using another log4j2 version. > It seems that relocation was dropped in possibly unintenionally in > CALCITE-4152. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4934) Deploy snapshot releases to Apache snapshot repo
[ https://issues.apache.org/jira/browse/CALCITE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457707#comment-17457707 ] Jacques Nadeau commented on CALCITE-4934: - This seems like it is working but we need to be able to access the GitHub secrets for deployment. Filed INFRA-22601 to accomplish that. > Deploy snapshot releases to Apache snapshot repo > > > Key: CALCITE-4934 > URL: https://issues.apache.org/jira/browse/CALCITE-4934 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > It would be helpful to be able to test out snapshot builds of Calcite without > having to build locally. It seems like CALCITE-351 originally implemented > this but it hasn't been happening for many years. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4934) Deploy snapshot releases to Apache snapshot repo
[ https://issues.apache.org/jira/browse/CALCITE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457703#comment-17457703 ] Jacques Nadeau commented on CALCITE-4934: - To test this, I'm going to use a build-test branch on Apache. I plan to delete the branch after testing is complete. > Deploy snapshot releases to Apache snapshot repo > > > Key: CALCITE-4934 > URL: https://issues.apache.org/jira/browse/CALCITE-4934 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > It would be helpful to be able to test out snapshot builds of Calcite without > having to build locally. It seems like CALCITE-351 originally implemented > this but it hasn't been happening for many years. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4934) Deploy snapshot releases to Apache snapshot repo
Jacques Nadeau created CALCITE-4934: --- Summary: Deploy snapshot releases to Apache snapshot repo Key: CALCITE-4934 URL: https://issues.apache.org/jira/browse/CALCITE-4934 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Assignee: Jacques Nadeau It would be helpful to be able to test out snapshot builds of Calcite without having to build locally. It seems like CALCITE-351 originally implemented this but it hasn't been happening for many years. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4928) Decouple Janino from RelMetadataQuery
[ https://issues.apache.org/jira/browse/CALCITE-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4928. - Fix Version/s: 1.29.0 Resolution: Fixed Merged in [f3c09365c2eecbb5198d9acbef638f76053a49a9|https://github.com/apache/calcite/commit/f3c09365c2eecbb5198d9acbef638f76053a49a9] > Decouple Janino from RelMetadataQuery > - > > Key: CALCITE-4928 > URL: https://issues.apache.org/jira/browse/CALCITE-4928 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 1h > Remaining Estimate: 0h > > * After discussion in CALCITE-4879 and CALCITE-4914, lets create a minimally > invasive version of CALCITE-4539 that allows decoupling of Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4851) Build gives lots of 'Execution optimizations have been disabled' warnings
[ https://issues.apache.org/jira/browse/CALCITE-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456836#comment-17456836 ] Jacques Nadeau commented on CALCITE-4851: - Thanks [~nobigo], this is much appreciated! This has been quite irritating for many of us since the 7.x upgrade. > Build gives lots of 'Execution optimizations have been disabled' warnings > - > > Key: CALCITE-4851 > URL: https://issues.apache.org/jira/browse/CALCITE-4851 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.27.0 > Environment: MacOS > Gradle 7.2 > Jdk 11 >Reporter: duan xiong >Assignee: duan xiong >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Lots of warnings about Execution optimizations have been disabled when build > CALCITE. For example: > {noformat} > Execution optimizations have been disabled for task ':babel:sourcesJar' to > ensure correctness due to the following reasons: > - Gradle detected a problem with the following location: > '/calcite/babel/build/javacc/javaCCMain'. > Reason: Task ':babel:sourcesJar' uses this output of task ':babel:javaCCMain' > without declaring an explicit or implicit dependency. > This can lead to incorrect results being produced, depending on what order > the tasks are executed. Please refer to > https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency > for more details about this problem.{noformat} > and > {noformat} > Gradle detected a problem with the following location: '/calcite'. Reason: > Task ':gitProps' uses this output of task ':babel:checkstyleTest' without > declaring an explicit or implicit dependency. > This can lead to incorrect results being produced, depending on what order > the tasks are executed. Please refer to > https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency > for more details about this problem.{noformat} > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (CALCITE-4914) Prep existing interfaces to support making RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau closed CALCITE-4914. --- Resolution: Abandoned Closing in preference to CALCITE-4928 > Prep existing interfaces to support making RelMetadataQuery abstract > > > Key: CALCITE-4914 > URL: https://issues.apache.org/jira/browse/CALCITE-4914 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Some field/constructors are currently exposed in such a way that abstracting > RelMetdataQuery must be done in two steps. This is the first step of marking > interfaces as deprecated and/or internal so that a later release can make > these abstract. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau closed CALCITE-4879. --- Resolution: Abandoned Closing in preference to CALCITE-4928 > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4929) Add default methods for getDef on metadata handlers
Jacques Nadeau created CALCITE-4929: --- Summary: Add default methods for getDef on metadata handlers Key: CALCITE-4929 URL: https://issues.apache.org/jira/browse/CALCITE-4929 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Right now, metadata handler implementations must implement a getDef() method. The values are singletons next to the interface definitions. This change proposes making these singletons be returned automatically by default methods rather than implementers needing to implement these known values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-4928) Decouple Janino from RelMetadataQuery
[ https://issues.apache.org/jira/browse/CALCITE-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4928: Description: * After discussion in CALCITE-4879 and CALCITE-4914, lets create a minimally invasive version of CALCITE-4539 that allows decoupling of Janino. (was: After discussion in CALCITE-4879 and CALCITE-4914, going to create a minimally invasive version of CALCITE-4539 that allows decoupling of Janino.) > Decouple Janino from RelMetadataQuery > - > > Key: CALCITE-4928 > URL: https://issues.apache.org/jira/browse/CALCITE-4928 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > * After discussion in CALCITE-4879 and CALCITE-4914, lets create a minimally > invasive version of CALCITE-4539 that allows decoupling of Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4928) Decouple Janino from RelMetadataQuery
Jacques Nadeau created CALCITE-4928: --- Summary: Decouple Janino from RelMetadataQuery Key: CALCITE-4928 URL: https://issues.apache.org/jira/browse/CALCITE-4928 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Assignee: Jacques Nadeau After discussion in CALCITE-4879 and CALCITE-4914, going to create a minimally invasive version of CALCITE-4539 that allows decoupling of Janino. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454151#comment-17454151 ] Jacques Nadeau commented on CALCITE-4879: - > Did you notice that we have a MetadataFactory that query the metadata using > direct java call(not based on Janino) ? Yes. Unfortunately, it isn't a substitute for RelMetadataQuery. RelMetadataQuery is used extensively throughout the codebase (including by numerous rules). Here is one concrete example: [ProjectJoinRemoveRule|https://github.com/apache/calcite/blob/d70583c4a8013f878457f82df6dffddd71875900/core/src/main/java/org/apache/calcite/rel/rules/ProjectJoinRemoveRule.java#L106]. One cannot use this rule without having an implementation of RelMetadataQuery. RelMetadataQuery is hard-coded to use Janino. This pattern repeats itself in a large number of places. There is no way to "replace" RelMetadataQuery with MetadataFactory in these situations (that I see). Additionally, MetadataFactory is deprecated on RelOptCluster, AbstractRelNode and isn't on the RelNode interface. The only implementation is also deprecated. (For comparison, [192|https://github.com/apache/calcite/search?q=RelMetadataQuery] Calcite files reference RelMetadataQuery, [4|https://github.com/apache/calcite/search?q=MetadataFactory] reference RelMetadataFactory and 3 of those 4 have references that are specifically deprecated.) Building a new system on MetadataFactory seems very much to neither solve the problem at hand nor be a good idea given its deprecation. Do you agree with 4 assertions above? > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453525#comment-17453525 ] Jacques Nadeau commented on CALCITE-4879: - As I have continued to think about this tonight, it struck me that one way to probably clean up a lot of this is to move the revision concept inside Janino's handlers rather than have them leak into RelMetdataQuery. At that point, RelMetadataQuery could avoid the coupling with Janino and move to only relying on RelMetadataProvider. If Janino defined handler decorators that handled the revision concepts, that could be contained entirely within the janino provider and no one else would need to know about revision. (the reason that janino is hardcoded is because RelMetadataQuery needs to know about revision). > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453519#comment-17453519 ] Jacques Nadeau commented on CALCITE-4879: - I'm wondering if there is a disconnect between the current state of the code and what the original intentions may have been. I'd like to state a few basic assertions to make sure we're working from the same foundation. Assertion #1: RelMetadataQuery is hardcoded to force the use of Janino. Assertion #2: RelMetadataQuery has a bunch of specific behavior implemented within it based on the concept of handler revision. Assertion #3: Handler revision isn't related to RelMetadataProvider and is a Janino-specific concept. Assertion #4: There are several fields exposed on the public API of RelMetadataQuery (via RelMetadataQueryBase) that are internal and shouldn't be exposed to consumers/users of RelMetdataQuery. Do people agree with these assertions? Once we are aligned on consistent foundation, we can further discuss possible changes. > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4879: --- Assignee: Jacques Nadeau > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4914) Prep existing interfaces to support making RelMetadataQuery abstract
Jacques Nadeau created CALCITE-4914: --- Summary: Prep existing interfaces to support making RelMetadataQuery abstract Key: CALCITE-4914 URL: https://issues.apache.org/jira/browse/CALCITE-4914 Project: Calcite Issue Type: Improvement Components: core Reporter: Jacques Nadeau Some field/constructors are currently exposed in such a way that abstracting RelMetdataQuery must be done in two steps. This is the first step of marking interfaces as deprecated and/or internal so that a later release can make these abstract. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452168#comment-17452168 ] Jacques Nadeau commented on CALCITE-4879: - > If people had been allowed to sub-class RelMetadataQuery they would have People can and do sub-class RelMetadataQuery in its entirety. Nothing in the code/construction of RelMetadataQuery or how RelOptCluster exposes a query supplier restricts that. I've actually done it when implementing a new AOT friendly RelMetadataQuery external to Calcite. > RelMetadataQuery is a facade. It doesn't contain state or behavior. I would disagree about it not containing state and behavior. It contains both. It holds state around current versions of each handler as well as thread local caches of janino compiled code and a cache of past metadata results. It also contains behavior including: null canonicalization, cache management and concepts around handler creation and revision. The latter is something that is very specific concept to a runtime compilation pattern that doesn't match well to an AOT pattern. > and we would have a mess today I think things are a bit of a mess now. This isn't a dig or a negative statement (and I'm not trying to start a flame war here). It's an observation after spending several weeks working with the existing code and building a complete compatible system (passes all the same tests as the existing one) that performs well and functions well in AOT environments. From my pov, there are many good things that exist: * it is relatively straightforward to add new metadata types (albeit verbose), * performance is good if you're in an environment that can support runtime compilation * Virtually all of planning-side consumption is limited to the a constrained portion of RelMetadataQuery that should be public. But there are also multiple real challenges: * Excessive runtime code that is not runtime specific/runtime compiled required * Bindings between components are brittle and implicit (javac/ide won't help identify many issues) * Several internal concepts are exposed as public on RelMetadataQuery (see the map, metadataProvider and THREAD_PROVIDER fields) * There is a large amount of deprecated code supporting old patterns heavily intertwined with the existing system * Major caching bugs (according to comments by others, I haven't validated) * Poor documentation * Debugging and understanding the code is quite difficult I agree that lack of freedom can be a feature. I don't think I'm altering the current state of freedom, just cleaning up and formalizing the freedom that already exists. > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (CALCITE-4841) Support decimal column type in CSV and File adapter
[ https://issues.apache.org/jira/browse/CALCITE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4841. - Fix Version/s: 1.29.0 Resolution: Fixed Resolved in [065e323da89ca73a46bec195efc7bd4019dc4f2f|https://github.com/apache/calcite/commit/065e323da89ca73a46bec195efc7bd4019dc4f2f] > Support decimal column type in CSV and File adapter > --- > > Key: CALCITE-4841 > URL: https://issues.apache.org/jira/browse/CALCITE-4841 > Project: Calcite > Issue Type: Improvement > Components: csv-adapter, file-adapter >Reporter: Louis Kuang >Priority: Minor > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, the csv adapter does not support the decimal/numeric column type > (see supported types in > [CsvFieldType|https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/file/src/main/java/org/apache/calcite/adapter/file/CsvFieldType.java#L35]. > Any type that is not supported will be interpreted by the > [CsvEnumerator|https://github.com/apache/calcite/blob/master/file/src/main/java/org/apache/calcite/adapter/file/CsvEnumerator.java] > as string. > When dealing with decimal numbers, the currently most appropriate type is > `double`. However, this is not accurate enough for financial data. This > feature request proposes adding a `decimal` column type that will be > implemented by the Java `BigDecimal` type (and by conversion in > [JavaToSqlTypeConversionRules|https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/core/src/main/java/org/apache/calcite/sql/type/JavaToSqlTypeConversionRules.java#L74] > be represented as a `Decimal` SQL type). This allow financial data to be > represented and computed more accurately (`BigDecimal` has higher precision > than `double`). > Please see sample implementation in PR. > Context: I am trying to leverage Calcite to add some SQL support to > [ledger|https://www.ledger-cli.org/] reporting. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4882) Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval
[ https://issues.apache.org/jira/browse/CALCITE-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441452#comment-17441452 ] Jacques Nadeau commented on CALCITE-4882: - For reference, I've confirmed that this new methodology works in both the JIT and in GraalVM native image with AOT compilation. > Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval > > > Key: CALCITE-4882 > URL: https://issues.apache.org/jira/browse/CALCITE-4882 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4882) Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval
[ https://issues.apache.org/jira/browse/CALCITE-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441432#comment-17441432 ] Jacques Nadeau commented on CALCITE-4882: - {quote}Could you post your code? {quote} Planning to, was just trying to get a microbenchmark working. Initial microbenchmark, will need help from others to verify that it is actually reloading everything in the (_with_compile variation). {{Benchmark Mode Cnt Score Error Units}} {{MetadataBenchmark.janino avgt 10 716.300 ± 21.170 ms/op}} {{MetadataBenchmark.janino_with_compile avgt 10 757.792 ± 27.464 ms/op}} {{MetadataBenchmark.lambda avgt 10 699.888 ± 29.359 ms/op}} {{MetadataBenchmark.lambda_with_construction avgt 10 751.197 ± 137.892 ms/op}} > Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval > > > Key: CALCITE-4882 > URL: https://issues.apache.org/jira/browse/CALCITE-4882 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4882) Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval
[ https://issues.apache.org/jira/browse/CALCITE-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441347#comment-17441347 ] Jacques Nadeau commented on CALCITE-4882: - Early Results for JdbcTest.testJoinFiveWay() ||Run Number||Janino||Lambda|| |1|19975|12010| |2|974|676| |3|916|625| |4|623|577| |5|617|553| |6|719|539| |7|760|541| |8|695|529| |9|622|517| |10|513|495| > Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval > > > Key: CALCITE-4882 > URL: https://issues.apache.org/jira/browse/CALCITE-4882 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4882) Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval
Jacques Nadeau created CALCITE-4882: --- Summary: Explore a LambdaMetadataFactory alternative to Janino for metadata retrieval Key: CALCITE-4882 URL: https://issues.apache.org/jira/browse/CALCITE-4882 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440074#comment-17440074 ] Jacques Nadeau commented on CALCITE-4879: - I suggest we deprecate constructors in 1.29 and then move to abstract as part of 1.30 > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CALCITE-4879) Make RelMedataQuery abstract
Jacques Nadeau created CALCITE-4879: --- Summary: Make RelMedataQuery abstract Key: CALCITE-4879 URL: https://issues.apache.org/jira/browse/CALCITE-4879 Project: Calcite Issue Type: Improvement Components: core Reporter: Jacques Nadeau The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction are great at separating how planners and rules consume metadata versus how metadata is produced. While details about how metadata is produced is (mostly) not leaked in the api of RelMetadataQuery, the class does assume that metadata will be produced via the current mechanisms surrounding RelMetadataProviders and MetadataHandlers. This ticket targets separating the production of metadata from the consumption interface, by making RelMetadataQuery abstract (either as an abstract class or as a interface) and moving the handler and provider specific implementations to an implementation of RelMetadataQuery. This will allow a broader breadth of experimentation to be undertaken. For example, one example people have been evaluating is whether a lambda based system would be easier to understand and debug, as performant and more AOT friendly than the existing systems of chains and janino compilation. To accomplish this task, the first step will be to deprecate the existing constructors and inform people to use a concrete subtype. Once deprecated, the actual logic that currently exists in RelMetadataQuery can be extracted into the concrete subtype and the base class can be made either abstract or an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4879) Make RelMetadataQuery abstract
[ https://issues.apache.org/jira/browse/CALCITE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4879: Summary: Make RelMetadataQuery abstract (was: Make RelMedataQuery abstract) > Make RelMetadataQuery abstract > -- > > Key: CALCITE-4879 > URL: https://issues.apache.org/jira/browse/CALCITE-4879 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Priority: Major > > The RelOptCluster.setMetadataQuerySupplier() and RelMedataQuery abstraction > are great at separating how planners and rules consume metadata versus how > metadata is produced. While details about how metadata is produced is > (mostly) not leaked in the api of RelMetadataQuery, the class does assume > that metadata will be produced via the current mechanisms surrounding > RelMetadataProviders and MetadataHandlers. This ticket targets separating the > production of metadata from the consumption interface, by making > RelMetadataQuery abstract (either as an abstract class or as a interface) and > moving the handler and provider specific implementations to an implementation > of RelMetadataQuery. This will allow a broader breadth of experimentation to > be undertaken. For example, one example people have been evaluating is > whether a lambda based system would be easier to understand and debug, as > performant and more AOT friendly than the existing systems of chains and > janino compilation. > To accomplish this task, the first step will be to deprecate the existing > constructors and inform people to use a concrete subtype. Once deprecated, > the actual logic that currently exists in RelMetadataQuery can be extracted > into the concrete subtype and the base class can be made either abstract or > an interface (depending on what seems most appropriate at the time). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4839) Remove remnants of ImmutableBeans post 1.28 release
[ https://issues.apache.org/jira/browse/CALCITE-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4839. - Resolution: Fixed Resolved in [bebe473fab2e242736614659ed6e5d04eeeb8bf5|https://github.com/apache/calcite/commit/bebe473fab2e242736614659ed6e5d04eeeb8bf5] > Remove remnants of ImmutableBeans post 1.28 release > --- > > Key: CALCITE-4839 > URL: https://issues.apache.org/jira/browse/CALCITE-4839 > Project: Calcite > Issue Type: Task >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This is final piece of work related to moving to Immutables. Once we release, > we need to go through and remove all the shims to support deprecation. This > includes deleting ImmutableBeans and the associated tests and annotations. We > also need to remove the deprecated Config interfaces (where they have new > names). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4836) Upgrade protobuf-java 3.6.1 -> 3.17.1
[ https://issues.apache.org/jira/browse/CALCITE-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426268#comment-17426268 ] Jacques Nadeau commented on CALCITE-4836: - Thanks [~nobigo]! > Upgrade protobuf-java 3.6.1 -> 3.17.1 > - > > Key: CALCITE-4836 > URL: https://issues.apache.org/jira/browse/CALCITE-4836 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.27.0 >Reporter: duan xiong >Assignee: duan xiong >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As CALCITE-4626 upgrade the protobuf-java to 3.17.1 to resolve the compile > warning. In order to keep the same version as it, upgrade this version in > CALCITE too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4836) Upgrade protobuf-java 3.6.1 -> 3.17.1
[ https://issues.apache.org/jira/browse/CALCITE-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4836. - Resolution: Fixed Resolved in [814ca45be67201d8433c72ae77b02bb872c727d7|https://github.com/apache/calcite/commit/814ca45be67201d8433c72ae77b02bb872c727d7] > Upgrade protobuf-java 3.6.1 -> 3.17.1 > - > > Key: CALCITE-4836 > URL: https://issues.apache.org/jira/browse/CALCITE-4836 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.27.0 >Reporter: duan xiong >Assignee: duan xiong >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As CALCITE-4626 upgrade the protobuf-java to 3.17.1 to resolve the compile > warning. In order to keep the same version as it, upgrade this version in > CALCITE too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426246#comment-17426246 ] Jacques Nadeau commented on CALCITE-4840: - I posted a patch of the changes I was thinking about. I'm not about boiling the ocean either... WRT to README vs README.md, I would think having only a README.md would be sufficient but haven't read bylaws lately. Out of curiosity, I took a look at the Spark 3.1.2 source distribution and it only has a README.md, not a README. Anyway, not the most important thing, I was just trying to improve small things along the way with fresh eyes where possible. If you're against any changes with this ticket and my patch, I'm fine to abandon both. > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: newbie > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4840: --- Assignee: Jacques Nadeau > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: newbie > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425877#comment-17425877 ] Jacques Nadeau commented on CALCITE-4840: - One other note, it's weird that there are two readmes and they have different information. Any rationale? I think most projects just have a single md (I don't have any numbers to back this up). > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Priority: Major > Labels: newbie > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425874#comment-17425874 ] Jacques Nadeau commented on CALCITE-4840: - You're right, I missed it. :) It could be the I'm the only one too impatient to find it but I bet others might miss it too. I'm inclined to post a patch adjusting the readme so the information is more hierarchical. > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Priority: Major > Labels: newbie > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4840: Labels: newbie (was: ) > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Priority: Major > Labels: newbie > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4840) Avatica readme for source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4840: Summary: Avatica readme for source release should have better build instructions (was: Avatica readme from source release should have better build instructions) > Avatica readme for source release should have better build instructions > --- > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Priority: Major > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4840) Avatica readme from source release should have better build instructions
[ https://issues.apache.org/jira/browse/CALCITE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4840: Summary: Avatica readme from source release should have better build instructions (was: Readme from source release clearly state how to build) > Avatica readme from source release should have better build instructions > > > Key: CALCITE-4840 > URL: https://issues.apache.org/jira/browse/CALCITE-4840 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Jacques Nadeau >Priority: Major > > Right now, trying to figure out how you should build from the source release > is too difficult (unless I missed something). To find the rather meagre > instructions, you have to: > # Open the README and see it says use README.md > # Open README.md and follow the link to the avatica home > # Click develop > # Click avatica development guide link > # Find and click "build from a source distribution" > Good luck to someone to find that path :) > When you get there, it states java 8 or later but trying to build with Java > 17 fails. For reference, jvm I was using: > {{% java --version}} > {{openjdk 17 2021-09-14 LTS}} > {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} > {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, > sharing)}} > > Ideally the readme (or an install.md or a single click) should include: > * Prerequisites to build (gradle 6.8.1 and Java 8?) > * How to check signature > * How to build & test > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4840) Readme from source release clearly state how to build
Jacques Nadeau created CALCITE-4840: --- Summary: Readme from source release clearly state how to build Key: CALCITE-4840 URL: https://issues.apache.org/jira/browse/CALCITE-4840 Project: Calcite Issue Type: Improvement Components: avatica Reporter: Jacques Nadeau Right now, trying to figure out how you should build from the source release is too difficult (unless I missed something). To find the rather meagre instructions, you have to: # Open the README and see it says use README.md # Open README.md and follow the link to the avatica home # Click develop # Click avatica development guide link # Find and click "build from a source distribution" Good luck to someone to find that path :) When you get there, it states java 8 or later but trying to build with Java 17 fails. For reference, jvm I was using: {{% java --version}} {{openjdk 17 2021-09-14 LTS}} {{OpenJDK Runtime Environment Corretto-17.0.0.35.2 (build 17+35-LTS)}} {{OpenJDK 64-Bit Server VM Corretto-17.0.0.35.2 (build 17+35-LTS, mixed mode, sharing)}} Ideally the readme (or an install.md or a single click) should include: * Prerequisites to build (gradle 6.8.1 and Java 8?) * How to check signature * How to build & test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4839) Remove remnants of ImmutableBeans post 1.28 release
Jacques Nadeau created CALCITE-4839: --- Summary: Remove remnants of ImmutableBeans post 1.28 release Key: CALCITE-4839 URL: https://issues.apache.org/jira/browse/CALCITE-4839 Project: Calcite Issue Type: Task Reporter: Jacques Nadeau Assignee: Jacques Nadeau This is final piece of work related to moving to Immutables. Once we release, we need to go through and remove all the shims to support deprecation. This includes deleting ImmutableBeans and the associated tests and annotations. We also need to remove the deprecated Config interfaces (where they have new names). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4830) Remove remaining uses of ImmutableBeans and deprecate
[ https://issues.apache.org/jira/browse/CALCITE-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4830. - Fix Version/s: 1.28.0 Resolution: Fixed Resolved in [3170c3f21687cf75045572844dcd51e19f41d40a|https://github.com/apache/calcite/commit/3170c3f21687cf75045572844dcd51e19f41d40a] Thanks to everyone for their assistance! > Remove remaining uses of ImmutableBeans and deprecate > - > > Key: CALCITE-4830 > URL: https://issues.apache.org/jira/browse/CALCITE-4830 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Follow-on work from CALCITE-4787 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4830) Remove remaining uses of ImmutableBeans and deprecate
[ https://issues.apache.org/jira/browse/CALCITE-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4830: Summary: Remove remaining uses of ImmutableBeans and deprecate (was: Move Cassandra module to use Immutables instead of ImmutableBeans) > Remove remaining uses of ImmutableBeans and deprecate > - > > Key: CALCITE-4830 > URL: https://issues.apache.org/jira/browse/CALCITE-4830 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Follow-on work from CALCITE-4787 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4835) Release Calcite 1.28.0
[ https://issues.apache.org/jira/browse/CALCITE-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425270#comment-17425270 ] Jacques Nadeau edited comment on CALCITE-4835 at 10/6/21, 11:27 PM: I'm writing a release note block for the changes I worked on. I'm storing it as a gist so I can do it in makdown as opposed to JIRA-ese. Happy to put it elsewhere if that is helpful. [https://gist.github.com/jacques-n/23c4060c3df5bfe5212c63ec5ba11667] was (Author: jnadeau): I'm writing a release note block for the changes I worked on. I'm storing it as a gist so I can do it in makdown as opposed to JIRA-ese. Happy to put it elsewhere it that is helpful. https://gist.github.com/jacques-n/23c4060c3df5bfe5212c63ec5ba11667 > Release Calcite 1.28.0 > -- > > Key: CALCITE-4835 > URL: https://issues.apache.org/jira/browse/CALCITE-4835 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.28.0 > > > Release Calcite 1.28.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4835) Release Calcite 1.28.0
[ https://issues.apache.org/jira/browse/CALCITE-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425270#comment-17425270 ] Jacques Nadeau commented on CALCITE-4835: - I'm writing a release note block for the changes I worked on. I'm storing it as a gist so I can do it in makdown as opposed to JIRA-ese. Happy to put it elsewhere it that is helpful. https://gist.github.com/jacques-n/23c4060c3df5bfe5212c63ec5ba11667 > Release Calcite 1.28.0 > -- > > Key: CALCITE-4835 > URL: https://issues.apache.org/jira/browse/CALCITE-4835 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.28.0 > > > Release Calcite 1.28.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4831) Cleanup Nullable rule annotations post fix for Immutables checker conflict bug
[ https://issues.apache.org/jira/browse/CALCITE-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau reassigned CALCITE-4831: --- Assignee: Jacques Nadeau > Cleanup Nullable rule annotations post fix for Immutables checker conflict bug > -- > > Key: CALCITE-4831 > URL: https://issues.apache.org/jira/browse/CALCITE-4831 > Project: Calcite > Issue Type: Task >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > Immutables has a small incompatibility issue when compiling cross-module > using the checker nullable annotations (works fine with > javax.annotation.Nullable). They are looking to fix it shortly. In the > meantime, we need to double-annotation the description field of > RelRule.Config to allow separate module Immutables to be correctly generated > with description as a nullable field. This requires a second annotation and > whitelisting RelRule in the autostyle application. > Pending on this ticket: > https://github.com/immutables/immutables/issues/1262 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4831) Cleanup Nullable rule annotations post fix for Immutables checker conflict bug
Jacques Nadeau created CALCITE-4831: --- Summary: Cleanup Nullable rule annotations post fix for Immutables checker conflict bug Key: CALCITE-4831 URL: https://issues.apache.org/jira/browse/CALCITE-4831 Project: Calcite Issue Type: Task Reporter: Jacques Nadeau Immutables has a small incompatibility issue when compiling cross-module using the checker nullable annotations (works fine with javax.annotation.Nullable). They are looking to fix it shortly. In the meantime, we need to double-annotation the description field of RelRule.Config to allow separate module Immutables to be correctly generated with description as a nullable field. This requires a second annotation and whitelisting RelRule in the autostyle application. Pending on this ticket: https://github.com/immutables/immutables/issues/1262 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4825) Update remaining core/main to use immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4825. - Fix Version/s: 1.28.0 Resolution: Fixed Fixed in [5824c54bbaa7cf527dd20ba85fb3b180219249e6|https://github.com/apache/calcite/commit/5824c54bbaa7cf527dd20ba85fb3b180219249e6] > Update remaining core/main to use immutables instead of ImmutableBeans > -- > > Key: CALCITE-4825 > URL: https://issues.apache.org/jira/browse/CALCITE-4825 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As part of CALCITE-4787, most of the core module was updated to use > Immutables. This is for the remaining parts of the non-test code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4830) Move Cassandra module to use Immutables instead of ImmutableBeans
Jacques Nadeau created CALCITE-4830: --- Summary: Move Cassandra module to use Immutables instead of ImmutableBeans Key: CALCITE-4830 URL: https://issues.apache.org/jira/browse/CALCITE-4830 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Assignee: Jacques Nadeau Follow-on work from CALCITE-4787 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4827) Refactor annotation processor tasks to be used across multiple modules
Jacques Nadeau created CALCITE-4827: --- Summary: Refactor annotation processor tasks to be used across multiple modules Key: CALCITE-4827 URL: https://issues.apache.org/jira/browse/CALCITE-4827 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Assignee: Jacques Nadeau Right now, only the core module runs the Immutable processor task. This ticket is to move that to a general place so that multiple modules may use the same code with minimal duplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4825) Update remaining core/main to use immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4825: Summary: Update remaining core/main to use immutables instead of ImmutableBeans (was: Update enumerables rules to use immutables instead of ImmutableBeans.EMPTY) > Update remaining core/main to use immutables instead of ImmutableBeans > -- > > Key: CALCITE-4825 > URL: https://issues.apache.org/jira/browse/CALCITE-4825 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > > As part of CALCITE-4787, most of the core module was updated to use > Immutables. This is for the remaining parts of the non-test code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424038#comment-17424038 ] Jacques Nadeau edited comment on CALCITE-4787 at 10/4/21, 6:21 PM: --- Resolved in [511eecdf5a0c68632f99c6b3e07ebbdd964c281d|https://github.com/apache/calcite/commit/511eecdf5a0c68632f99c6b3e07ebbdd964c281d] was (Author: jnadeau): Resolved in [d70583c4a8013f878457f82df6dffddd71875900|https://github.com/apache/calcite/commit/d70583c4a8013f878457f82df6dffddd71875900] > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4826) annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij
[ https://issues.apache.org/jira/browse/CALCITE-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4826: Fix Version/s: 1.28.0 > annotationProcessorMain task is misconfigured in core gradle build, causing > sync to fail in Intellij > > > Key: CALCITE-4826 > URL: https://issues.apache.org/jira/browse/CALCITE-4826 > Project: Calcite > Issue Type: Bug >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10m > Remaining Estimate: 0h > > After CALCITE-4787, Intellij will fail when syncing with: > {quote}{{A problem was found with the configuration of task > ':core:annotationProcessorMain' (type 'JavaCompile').}} > {{> No value has been specified for property 'destinationDirectory'.}} > {quote} > > This property was removed and tested in the Intellij before the merge but > apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4826) annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij
[ https://issues.apache.org/jira/browse/CALCITE-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424091#comment-17424091 ] Jacques Nadeau commented on CALCITE-4826: - I force pushed the fix as part of CALCITE-4787 > annotationProcessorMain task is misconfigured in core gradle build, causing > sync to fail in Intellij > > > Key: CALCITE-4826 > URL: https://issues.apache.org/jira/browse/CALCITE-4826 > Project: Calcite > Issue Type: Bug >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After CALCITE-4787, Intellij will fail when syncing with: > {quote}{{A problem was found with the configuration of task > ':core:annotationProcessorMain' (type 'JavaCompile').}} > {{> No value has been specified for property 'destinationDirectory'.}} > {quote} > > This property was removed and tested in the Intellij before the merge but > apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4826) annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij
[ https://issues.apache.org/jira/browse/CALCITE-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4826. - Resolution: Duplicate Fixed in CALCITE-4787 > annotationProcessorMain task is misconfigured in core gradle build, causing > sync to fail in Intellij > > > Key: CALCITE-4826 > URL: https://issues.apache.org/jira/browse/CALCITE-4826 > Project: Calcite > Issue Type: Bug >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After CALCITE-4787, Intellij will fail when syncing with: > {quote}{{A problem was found with the configuration of task > ':core:annotationProcessorMain' (type 'JavaCompile').}} > {{> No value has been specified for property 'destinationDirectory'.}} > {quote} > > This property was removed and tested in the Intellij before the merge but > apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4826) annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij
[ https://issues.apache.org/jira/browse/CALCITE-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424081#comment-17424081 ] Jacques Nadeau commented on CALCITE-4826: - Do you both think I should just force push the fix? If so, I'll do so. It is a single line fix. I just hate force-pushing master on principle. > annotationProcessorMain task is misconfigured in core gradle build, causing > sync to fail in Intellij > > > Key: CALCITE-4826 > URL: https://issues.apache.org/jira/browse/CALCITE-4826 > Project: Calcite > Issue Type: Bug >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After CALCITE-4787, Intellij will fail when syncing with: > {quote}{{A problem was found with the configuration of task > ':core:annotationProcessorMain' (type 'JavaCompile').}} > {{> No value has been specified for property 'destinationDirectory'.}} > {quote} > > This property was removed and tested in the Intellij before the merge but > apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4826) annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij
[ https://issues.apache.org/jira/browse/CALCITE-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4826: Summary: annotationProcessorMain task is misconfigured in core gradle build, causing sync to fail in Intellij (was: Fix Intellij annotation processing failure post CALCITE-4787) > annotationProcessorMain task is misconfigured in core gradle build, causing > sync to fail in Intellij > > > Key: CALCITE-4826 > URL: https://issues.apache.org/jira/browse/CALCITE-4826 > Project: Calcite > Issue Type: Bug >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After CALCITE-4787, Intellij will fail when syncing with: > {quote}{{A problem was found with the configuration of task > ':core:annotationProcessorMain' (type 'JavaCompile').}} > {{> No value has been specified for property 'destinationDirectory'.}} > {quote} > > This property was removed and tested in the Intellij before the merge but > apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4826) Fix Intellij annotation processing failure post CALCITE-4787
Jacques Nadeau created CALCITE-4826: --- Summary: Fix Intellij annotation processing failure post CALCITE-4787 Key: CALCITE-4826 URL: https://issues.apache.org/jira/browse/CALCITE-4826 Project: Calcite Issue Type: Bug Reporter: Jacques Nadeau Assignee: Jacques Nadeau After CALCITE-4787, Intellij will fail when syncing with: {quote}{{A problem was found with the configuration of task ':core:annotationProcessorMain' (type 'JavaCompile').}} {{> No value has been specified for property 'destinationDirectory'.}} {quote} This property was removed and tested in the Intellij before the merge but apparently was tested incorrectly. Adding it back fixes the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424059#comment-17424059 ] Jacques Nadeau commented on CALCITE-4787: - > When you mark a bug fixed please set the fix-version. Also use a hyperlink to >the github commit. Fixed, thanks for the reminders! > Are you going to do the follow-up tasks, such as CALCITE-4825, Yep. Working on that one right now. > and if so, will this be done before release 1.28? A half-done breaking change > may be difficult to explain in the release notes. My goal is to get as many as possible done before the release (ideally all). That being said, given the way that the patch ultimately turned out, I don't think there is a global breaking change change here that requires that this be all or nothing in a particular release. Mainly, my thinking is we shouldn't mark ImmutableBeans deprecated until no more Calcite code is using it. I'd like to get there before 1.28 but will have to see if I hit any snags as I work through the rest of the codebase. > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424038#comment-17424038 ] Jacques Nadeau edited comment on CALCITE-4787 at 10/4/21, 5:27 PM: --- Resolved in [d70583c4a8013f878457f82df6dffddd71875900|https://github.com/apache/calcite/commit/d70583c4a8013f878457f82df6dffddd71875900] was (Author: jnadeau): Resolved in d70583c4a8013f878457f82df6dffddd71875900 > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4787: Fix Version/s: 1.28.0 > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4825) Update enumerables rules to use immutables instead of ImmutableBeans.EMPTY
Jacques Nadeau created CALCITE-4825: --- Summary: Update enumerables rules to use immutables instead of ImmutableBeans.EMPTY Key: CALCITE-4825 URL: https://issues.apache.org/jira/browse/CALCITE-4825 Project: Calcite Issue Type: Improvement Reporter: Jacques Nadeau Assignee: Jacques Nadeau As part of CALCITE-4787, most of the core module was updated to use Immutables. This is for the remaining parts of the non-test code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved CALCITE-4787. - Resolution: Fixed Resolved in d70583c4a8013f878457f82df6dffddd71875900 > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4787) Move core to use Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4787: Summary: Move core to use Immutables instead of ImmutableBeans (was: Evaluate use of Immutables instead of ImmutableBeans) > Move core to use Immutables instead of ImmutableBeans > - > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4787) Evaluate use of Immutables instead of ImmutableBeans
[ https://issues.apache.org/jira/browse/CALCITE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated CALCITE-4787: Description: In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] was discussed as an alternative to a custom implementation. This ticket is to evaluate the impact to the codebase of changing. Ideally, introduction of immutables would both add flexibility and reduce the amount of code associated with these classes. Immutables works via annotation processor which means that it is should be relatively seamless to build systems and IDEs. The switch would also make it easier to work with these objects types in the context of aot compilation tools like GraalVM. This initial task covers key classes in the core module. Will open up follow-on tickets for other locations. was: In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] was discussed as an alternative to a custom implementation. This ticket is to evaluate the impact to the codebase of changing. Ideally, introduction of immutables would both add flexibility and reduce the amount of code associated with these classes. Immutables works via annotation processor which means that it is should be relatively seamless to build systems and IDEs. The switch would also make it easier to work with these objects types in the context of aot compilation tools like GraalVM. > Evaluate use of Immutables instead of ImmutableBeans > > > Key: CALCITE-4787 > URL: https://issues.apache.org/jira/browse/CALCITE-4787 > Project: Calcite > Issue Type: Improvement >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > > In the creation of CALCITE-3328, [Immutables|https://immutables.github.io/] > was discussed as an alternative to a custom implementation. This ticket is to > evaluate the impact to the codebase of changing. Ideally, introduction of > immutables would both add flexibility and reduce the amount of code > associated with these classes. > Immutables works via annotation processor which means that it is should be > relatively seamless to build systems and IDEs. > The switch would also make it easier to work with these objects types in the > context of aot compilation tools like GraalVM. > > This initial task covers key classes in the core module. Will open up > follow-on tickets for other locations. -- This message was sent by Atlassian Jira (v8.3.4#803005)