[
https://issues.apache.org/jira/browse/APEXMALHAR-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278850#comment-15278850
]
Ilya Ganelin edited comment on APEXMALHAR-1818 at 5/10/16 8:40 PM:
-------------------------------------------------------------------
Based on our initial discussion, there are a number of key steps in unifying
Calcite with Apex
**SQL Translation**
First, we need a method to translate a SQL query representation into an Apex
DAG.
1. SQL will be translated by Calcite into a relational algebra representation
(e.g. JSON)
2. Malhar translates provided schema definitions (either dynamically
communicated or statically specified) into appropriately structured/typed
input/output operators in the DAG and output as a JSON config
3. Malhar translates the algebraic representation into an Apex/Malhar DAG
representation output (JSON). This DAG can be considered the “internal” DAG
between the input and output operators defined by the schema
**Testing**
The Calcite project is presently defining a new testing framework which will
allow for offline out-of-band testing of frameworks that work with Calcite.
Testing wth the above translation process is straightforward:
1. Compare the JSON relational algebra representation against the expected
representation
2. Compare the JSON generated from the schema translation against the known
good translation
3. Compare the JSON generated by the SQL to DAG translation against the
expected DAG JSON representation.
The above approach should allow us to create a well modularized integration
that can be readily tested and supports dynamic and flexible reconfiguration.
[~julianhyde] [~thw] Please review and let me know if I left anything out.
was (Author: ilganeli):
Based on our initial discussion, there are a number of key steps in unifying
Calcite with Apex
SQL Translation
First, we need a method to translate a SQL query representation into an Apex
DAG.
1. SQL will be translated by Calcite into a relational algebra representation
(e.g. JSON)
2. Malhar translates provided schema definitions (either dynamically
communicated or statically specified) into appropriately structured/typed
input/output operators in the DAG and output as a JSON config
3. Malhar translates the algebraic representation into an Apex/Malhar DAG
representation output (JSON). This DAG can be considered the “internal” DAG
between the input and output operators defined by the schema
Testing
The Calcite project is presently defining a new testing framework which will
allow for offline out-of-band testing of frameworks that work with Calcite.
Testing wth the above translation process is straightforward:
1. Compare the JSON relational algebra representation against the expected
representation
2. Compare the JSON generated from the schema translation against the known
good translation
3. Compare the JSON generated by the SQL to DAG translation against the
expected DAG JSON representation.
The above approach should allow us to create a well modularized integration
that can be readily tested and supports dynamic and flexible reconfiguration.
[~julianhyde] [~thw] Please review and let me know if I left anything out.
> Integrate Calcite to support SQL
> --------------------------------
>
> Key: APEXMALHAR-1818
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-1818
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Components: query operators
> Reporter: Amol
> Assignee: Amol
> Labels: roadmap
>
> Once we have ability to generate a subdag, we should take a look at
> integrating Calcite into Apex. The operator that enables populate DAG, should
> use Calcite to generate the DAG, given a SQL query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
