[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037796#comment-15037796 ] Timo Walther commented on FLINK-2099: - I will soon present my SQL prototype. However, I'm looking for more complex ITCases. Do you know any suited benchmarks I can use for that? It would be great if the benchmark contains input tables (e.g. as CSV files), complex SQL queries and the result (also as CSV files). Actually, TPC-H would be perfect but does not has the right license if I remember correctly. Do you know an open source alternative? > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045170#comment-15045170 ] Ovidiu Marcu commented on FLINK-2099: - Hi Timo, I would be interested in the SQL prototype. What are your objectives for this presentation? Will it be available soon? This is a very important feature, can you give more details about it? Thank you! > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045519#comment-15045519 ] Timo Walther commented on FLINK-2099: - Hi Ovidiu, thanks for your interest. Actually I wanted to present my prototype in 1-2 weeks, after a large refactoring. If you want to have an early preview you can have a look into my branch https://github.com/twalthr/flink/tree/FlinkSQL. I'm using Apache Calcite to parse, validate, convert and optimize a SQL query. After optimization the Calcite tree nodes (logical repr.) are converted to Table API tree nodes (phsycial repr.). At the moment I'm supporting every feature that the Table API supports (see https://github.com/twalthr/flink/blob/FlinkSQL/flink-staging/flink-table/src/test/scala/org/apache/flink/api/table/sql/SqlSelectTest.scala). However, the Table API is not feature complete yet. There are some important things missing, which is why the SQL API does not support it yet: - complete NULL support - sorting - outer joins - custom functions If you would like to contribute, you are very welcome! > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046707#comment-15046707 ] Ovidiu Marcu commented on FLINK-2099: - Hi Timo, Thank you for your detailed response. For the benchmark you want to use, I am not sure if TPC-H is the best to follow, as the new benchmark TPC-DS (next generation decision support) is developed to handle the deficiencies of TPC-H (reference here http://www.tpc.org/tpcds/presentations/The_Making_of_TPCDS.pdf) I would like to help/contribute, I will look into your branch and come back to you. I think this is one the most important features of Flink, I don't know why Flink guys don't want to increase its priority and give more attention. Best regards, Ovidiu > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046761#comment-15046761 ] Fabian Hueske commented on FLINK-2099: -- Hi Ovidiu, thanks for your interest in Flink and welcome to the community! It would be awesome if you'd start to contribute. Support for TPC-D would definitely be very nice, but I think TPC-H is a good starting point because it is less complex (22 vs. 99 queries) and covers already quite a lot of the SQL space. Regarding the priority of features, my opinion is that there is an abundance of SQL-on-Hadoop solutions but a need for stream processors with a good feature set. I (and other members of the community) believe that this is the area where Flink can provide the best benefit to users. Of course support for SQL is an awesome feature to have for Flink and I am very happy that people are working on it. Best, Fabian > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046798#comment-15046798 ] Timo Walther commented on FLINK-2099: - [~ovidiumarcu] It would be very nice to have someone else helping me with that task. If the main issues of the Table API are solved, I think I can move the SQL code to flink-contrib or flink-staging. So that more people can work on it and open PR against it. [~fhueske] You are right. Supporting TPC-H queries first was also my initial thought because of less and simpler queries. However, we still have the license problem with generated data (see FLINK-705). I think we should use TPC-DS because there is already an Apache-licensed generator/validator tool: https://github.com/julianhyde/tpcds It is in Maven central I can thus easily integrated for ITCases. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[
https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048135#comment-15048135
]
Ovidiu Marcu commented on FLINK-2099:
-
I've just gave a short look.
After merging FLINK-2961 and making a little change in
flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/sql/calcite/RexToExpr.scala
(case DATE => Literal(literal.getValue3, BasicTypeInfo.DATE_TYPE_INFO) I have:
val actual = sqlTestUtil.translator.translate("""
SELECT *
FROM TABLE5
WHERE FIELD2 < DATE '2015-10-31'""")
println(actual) //->
Expression(Filter(Root(org.apache.flink.api.java.operators.MapOperator@27ad4dbe,ArraySeq((FIELD1,Integer),
(FIELD2,String))), 'FIELD2.cast(Date) < 16739))
How can I filter a Date with Table api?
In SqlTestUtil I have
val table5 = env.fromElements((1, "2015-10-31"), (2, "2015-10-30")).as('FIELD1,
'FIELD2)
tableRegistry.registerTable("TABLE5", table5)
All other tests are ok (well, except Ignored ones).
> Add a SQL API
> -
>
> Key: FLINK-2099
> URL: https://issues.apache.org/jira/browse/FLINK-2099
> Project: Flink
> Issue Type: New Feature
> Components: Table API
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> From the mailing list:
> Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best
> approach would be to leverage that and build a SQL parser (maybe together
> with a logical optimizer) on top of the Table API. Parser (and optimizer)
> could be built using Apache Calcite which is providing exactly this.
> Since the Table API is still a fairly new component and not very feature
> rich, it might make sense to extend and strengthen it before putting
> something major on top.
> Ted: It would also be relatively simple (I think) to retarget drill to Flink
> if
> Flink doesn't provide enough typing meta-data to do traditional SQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048439#comment-15048439 ] Timo Walther commented on FLINK-2099: - Cool, that you already checked out and tested my code. You already found a feature that is missing in the Table API: Proper Date comparison support. I have opened an issue for that (see FLINK-3152). I can assign it to you, if you like this could be your first contribution. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048601#comment-15048601 ] Ovidiu Marcu commented on FLINK-2099: - Thanks, I will look into FLINK-3152. Can you please make an umbrella task to link all SQL jira issues currently opened? I am also investigating ways to use the benchmark TPC-DS. This will raise other missing features probably. Regarding Table API this is all the docs I can find https://ci.apache.org/projects/flink/flink-docs-release-0.10/libs/table.html , maybe there is something else I'm missing and you can point out? Thanks. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048732#comment-15048732 ] Timo Walther commented on FLINK-2099: - We can use this issue as an umbrella task for all SQL requirements. I will link related issues. At the moment https://ci.apache.org/projects/flink/flink-docs-master/libs/table.html is the latest documentation. I think the easiest way to figure out the internals is to look into the Table API code and its examples (with a debugger). > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076096#comment-15076096 ] Stephan Ewen commented on FLINK-2099: - Started a simple design/architecture doc a while. May be helpful in fleshing out what should be done and how, and give contributors a simple introduction. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098675#comment-15098675 ] chenliang613 commented on FLINK-2099: - Ya, TPC-DS should be widely used benchmark for SQL on Hadoop solutions. As my experience, supports the SQL syntax would be not big problem, but the performance would be the major challenges, please consider it more. Thanks Liang > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > Labels: requires-design-doc > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266458#comment-15266458 ] Fabian Hueske commented on FLINK-2099: -- The current master branch supports a subset of SQL already and the features linked to this issue feel a bit arbitrary. Even if all these issues are fixed, we won't support the full set and cannot claim "SQL support is done". Therefore, I propose to close this issue. We will of course continue to extend the support for SQL features, but I don't think we need this issue to track progress. What do you think [~twalthr]? > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > Labels: requires-design-doc > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266493#comment-15266493 ] Timo Walther commented on FLINK-2099: - Yes, I think we can close this issue. Basic support of SQL is implemented. The remaining features are implemented incrementally. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > Labels: requires-design-doc > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[
https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580481#comment-14580481
]
Timo Walther commented on FLINK-2099:
-
I had a private discussion with [~rmetzger], [~StephanEwen], [~aljoscha] and
[~fhueske].
Where we came to consensus (correct me if I'm wrong or missed something):
- Build SQL API on top of Table API
- Use Apache Calcite for SQL parsing and logical optimization, without
providing schema information
- Let Table API's optimizer do basic optimizations so far and maybe use
Calcites relational algebra optimizer in the future
Meanwhile I made myself familiar with Apache Calcite and could not found a way
how we can do logical optimization without schema information. The question is
now: Do we really want to "connect" the SQL parse tree directly with the Table
API? Wouldn't it make sense to integrate the SQL into the Table API and use
schema information so that Calcite can optimize SQL completely (resolve
problems with correlated subqueries etc.)?
>From a users perspective it could look like this:
{code}
TableEnvironment tableEnv = new TableEnvironment();
DataSet input = env.fromElements(
new WC("Hello", 1),
new WC("Ciao", 1),
new WC("Hello", 1));
Table table = tableEnv.fromDataSet(input, "MYTABLE"); // registers the table
schema in TableEnvironment
table.select("...") // simple expressions with Table API
Table resultTable = tableEnv.sql("SELECT name FROM MYTABLE") // complex
expressions with SQL
{code}
> Add a SQL API
> -
>
> Key: FLINK-2099
> URL: https://issues.apache.org/jira/browse/FLINK-2099
> Project: Flink
> Issue Type: New Feature
> Components: Table API
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> From the mailing list:
> Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best
> approach would be to leverage that and build a SQL parser (maybe together
> with a logical optimizer) on top of the Table API. Parser (and optimizer)
> could be built using Apache Calcite which is providing exactly this.
> Since the Table API is still a fairly new component and not very feature
> rich, it might make sense to extend and strengthen it before putting
> something major on top.
> Ted: It would also be relatively simple (I think) to retarget drill to Flink
> if
> Flink doesn't provide enough typing meta-data to do traditional SQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[
https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581644#comment-14581644
]
Fabian Hueske commented on FLINK-2099:
--
That looks nice. :-)
I would make the registration a separate method on the TableEnvironment to
reduce duplicated methods and allow the registration of derived tables. We
could also add a method to register a full HCatalog and to get registered
tables such as
{code}
DataSet invoices = ...
TableEnvironment tEnv = ...
Table invoicesT = tEnv.fromDataSet(invoices);
tEnv.registerHCatalog(hCatConf);
tEnv.registerTable(invoicesT, "invoices")
Table users = tEnv.getTable("users"); // from HCatalog
{code}
> Add a SQL API
> -
>
> Key: FLINK-2099
> URL: https://issues.apache.org/jira/browse/FLINK-2099
> Project: Flink
> Issue Type: New Feature
> Components: Table API
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> From the mailing list:
> Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best
> approach would be to leverage that and build a SQL parser (maybe together
> with a logical optimizer) on top of the Table API. Parser (and optimizer)
> could be built using Apache Calcite which is providing exactly this.
> Since the Table API is still a fairly new component and not very feature
> rich, it might make sense to extend and strengthen it before putting
> something major on top.
> Ted: It would also be relatively simple (I think) to retarget drill to Flink
> if
> Flink doesn't provide enough typing meta-data to do traditional SQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2099) Add a SQL API
[ https://issues.apache.org/jira/browse/FLINK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595886#comment-14595886 ] Stephan Ewen commented on FLINK-2099: - +1, very nice idea We can still have a way to transform a DataSet into a Table without automatically registering it under a name. > Add a SQL API > - > > Key: FLINK-2099 > URL: https://issues.apache.org/jira/browse/FLINK-2099 > Project: Flink > Issue Type: New Feature > Components: Table API >Reporter: Timo Walther >Assignee: Timo Walther > > From the mailing list: > Fabian: Flink's Table API is pretty close to what SQL provides. IMO, the best > approach would be to leverage that and build a SQL parser (maybe together > with a logical optimizer) on top of the Table API. Parser (and optimizer) > could be built using Apache Calcite which is providing exactly this. > Since the Table API is still a fairly new component and not very feature > rich, it might make sense to extend and strengthen it before putting > something major on top. > Ted: It would also be relatively simple (I think) to retarget drill to Flink > if > Flink doesn't provide enough typing meta-data to do traditional SQL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
