[ https://issues.apache.org/jira/browse/IGNITE-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Orlov updated IGNITE-15832: -------------------------------------- Description: h3. Motivation The current SQL engine is powerful enough to handle distributed queries of arbitrary complexity. But to unleash its full potential, we need to provide users with ability to run SQL queries against custom storages (like queries against CSV file). h3. Proposal In order to achieve the goal we need to provide API classes and do a number of integration points in current codebase. The suggested API interface should look like this: {code:java} public interface SqlExtensionPlugin { String name(); void init( SchemaUpdateListener schemaUpdateListener ); Set<? extends RelOptRule> getOptimizerRules(PlannerPhase phase); <Row> NodeImplementor<Row> implementor(); ColocationGroup colocationGroup(RelNode node); interface NodeImplementor<Row> { <T extends RelNode> Node<Row> implement(ExecutionContext<Row> ctx, T node); } interface SchemaUpdateListener { void onSchemaUpdated(Schema schema); } } {code} h4. Details Before running queries against custom storage, user should complete follow steps: 1) Implement SqlExtensionPlugin 1.1) Notify schema manager about schemas and tables handled by current plugin. The schema manager caches provided schema, so to publish any updates to the shema, the user should invoke `onSchemaUpdated` every time. 1.2) Provide a list of rules to inject in the optimizer. For simple case of external storage no rules actually required. 1.3) Provide an implementer for every injected physical node. 1.4) Implement a `colocationGroup` which answering a question `which parts of the table resides on which nodes` Details of integration point are covered in subtasks. There is a [POC|https://github.com/apache/ignite-3/compare/main...gridgain:sql-ext-wip] that could be referenced during implementation of current task. was: h3. Motivation The current SQL engine is powerful enough to handle distributed queries of arbitrary complexity. But to unleash its full potential, we need to provide users with ability to run SQL queries against custom storages (like queries against CSV file). h3. Proposal In order to achieve the goal we need to provide API classes and do a number of integration points in current codebase. The suggested API interface should look like this: {code:java} public interface SqlExtensionPlugin { String name(); void init( SchemaUpdateListener schemaUpdateListener ); Set<? extends RelOptRule> getOptimizerRules(PlannerPhase phase); <Row> NodeImplementor<Row> implementor(); ColocationGroup colocationGroup(RelNode node); interface NodeImplementor<Row> { <T extends RelNode> Node<Row> implement(ExecutionContext<Row> ctx, T node); } interface SchemaUpdateListener { void onSchemaUpdated(Schema schema); } } {code} Details of integration point are covered in subtask. There is a [POC|https://github.com/apache/ignite-3/compare/main...gridgain:sql-ext-wip] that could be referenced during implementation of current task > Calcite. SQL Extension API > -------------------------- > > Key: IGNITE-15832 > URL: https://issues.apache.org/jira/browse/IGNITE-15832 > Project: Ignite > Issue Type: Task > Reporter: Konstantin Orlov > Assignee: Konstantin Orlov > Priority: Major > Labels: calcite, calcite3-required, ignite-3 > > h3. Motivation > The current SQL engine is powerful enough to handle distributed queries of > arbitrary complexity. But to unleash its full potential, we need to provide > users with ability to run SQL queries against custom storages (like queries > against CSV file). > h3. Proposal > In order to achieve the goal we need to provide API classes and do a number > of integration points in current codebase. The suggested API interface should > look like this: > {code:java} > public interface SqlExtensionPlugin { > String name(); > void init( > SchemaUpdateListener schemaUpdateListener > ); > Set<? extends RelOptRule> getOptimizerRules(PlannerPhase phase); > <Row> NodeImplementor<Row> implementor(); > ColocationGroup colocationGroup(RelNode node); > interface NodeImplementor<Row> { > <T extends RelNode> Node<Row> implement(ExecutionContext<Row> ctx, T > node); > } > interface SchemaUpdateListener { > void onSchemaUpdated(Schema schema); > } > } > {code} > h4. Details > Before running queries against custom storage, user should complete follow > steps: > 1) Implement SqlExtensionPlugin > 1.1) Notify schema manager about schemas and tables handled by current > plugin. The schema manager caches provided schema, so to publish any updates > to the shema, the user should invoke `onSchemaUpdated` every time. > 1.2) Provide a list of rules to inject in the optimizer. For simple case of > external storage no rules actually required. > 1.3) Provide an implementer for every injected physical node. > 1.4) Implement a `colocationGroup` which answering a question `which parts of > the table resides on which nodes` > Details of integration point are covered in subtasks. > There is a > [POC|https://github.com/apache/ignite-3/compare/main...gridgain:sql-ext-wip] > that could be referenced during implementation of current task. -- This message was sent by Atlassian Jira (v8.3.4#803005)