Hi Rui,
Your sum-up is precisely what I care about. > 1. Reuse Calcite adaptors and combine that with your parser to parse queries. First, that is what @Michael described, doesn't it? The coding work to solve it makes sense to me. Thanks. > 2. Convert results from 1. to RelNode and let Calcite optimize, then execute > based on Enumerable implementations. Second, that is my concern now. Yep, we need a trigger or entrance to make Calcite use the custom adapter with the external parsed result (Precisely speaking, the RelNode converted from the third-party parsed result). I guess it is Calcite Driver or Calcite connection is that `trigger`. However, it can not use the output from point 1. > write code to execute the enumerable tree (this part of code is inside Calcite connection, but Calcite connection won't let you use your own parser) That way, maybe we need to rewrite a `Calcite connection` to execute the enumerable tree? How about implementing `SqlAbstractParserImpl` and configure it in JDBC props? Thanks for your time. Best wishes, Trista Juan Pan (Trista) Senior DBA & PMC of Apache ShardingSphere E-mail: panj...@apache.org On 11/24/2020 06:16,Rui Wang<amaliu...@apache.org> wrote: I think Michael has said points that I can say. Just try to understand your problem after reads existing threads: Sounds like, you need to do two things: 1. Reuse Calcite adaptors and combine that with your parser to parse queries. 2. Convert results from 1. to RelNode and let Calcite optimize, then execute based on Enumerable implementations. If my understanding so far is correct, I am thinking Calcite does not have a simple API to allow you do 2. My understanding is you will need to build something by yourself: a. write code to convert results of 1. to RelNode, make sure to set up Enumerable conventions to produce Enumerable backed nodes. b. write code to execute the enumerable tree (this part of code is inside Calcite connection, but Calcite connection won't let you use your own parser) -Rui On Mon, Nov 23, 2020 at 4:10 AM Michael Mior <mm...@apache.org> wrote: There is nothing stopping you from using adapters with SQL queries you have parsed yourself. You simply need to assign the appropriate convention to each table scan in the RelNode tree you pass into the optimizer. However, if the reason for using your own parser is to be able to have as broad support for different SQL queries as possible, I suggest you look at Calcite's Babel parser. It extends the default parser to add broader support for other dialects of SQL. -- Michael Mior mm...@apache.org Le lun. 23 nov. 2020 à 01:54, Juan Pan <panj...@apache.org> a écrit : Hi JiaTao, Very appreciated your share. Actually, what I am confused about is how to make Calcite custom adaptor works with other parsers. For example, I use a non-Calcite parser to get the parsed result and transform them into RelNode to tell Calcite, Hi, please use this RelNode for the rest handling. But I still implement a custom adaptor and wish Calcite can adopt them. If I call Calcite JDBC, like `Driver.getConnection(Calcite_Conn)`, which will bring Calcite parser to parser SQL instead of my own. : ( Is there any approach to make Calcite call the custom adapter and third-party parser? Best wishes, Trista Juan Pan (Trista) Senior DBA & PMC of Apache ShardingSphere E-mail: panj...@apache.org On 11/23/2020 14:38,JiaTao Tao<taojia...@gmail.com> wrote: Hi Juan Pan As I said, you can archive this by "If you have to do this, you can either generate SqlNode with Antlr OR transform your own AST tree to RelNode, you can take a look at org.apache.calcite.sql2rel.SqlToRelConverter.", in fact, hive does the same thing, you can take a look, it uses its own AST tree to generate a RelNode tree. Regards! Aron Tao Juan Pan <panj...@apache.org> 于2020年11月23日周一 下午1:04写道: Hi JiaTao, The reason we want to bypass Calcite parsing mainly contains two points. First, as you said, we want to have a better query efficiency by only parsing SQL one time. But from what you said, it looks like not a big deal. Second, I am a bit concerned about the SQL supported capacity of Calcite. [1] shows me the all supported SQLs. Is that in line with SQL92 or MySQL 5.x? Currently, ShardingSphere parser has almost complete support for MySQL 8.0 and PostgreSQL, and basic support for SQLServer, Oracle, SQL92 [2] (As a distributed Database middleware ecosystem, we have to do so). Therefore, if we use Calcite parser, maybe we can not help users handle some of the SQLs (Unsure). Could you give me some hints to bypass the parsing of Calcite? Or maybe we can not reach that goal? Much appreciated your any points or reply. : ) Regards, Trista [1] https://calcite.apache.org/docs/reference.html [2] https://shardingsphere.apache.org/document/current/en/features/sharding/principle/parse Juan Pan (Trista) Senior DBA & PMC of Apache ShardingSphere E-mail: panj...@apache.org On 11/22/2020 16:17,JiaTao Tao<taojia...@gmail.com> wrote: In fact, parse twice's impact is little, in Apache Kylin, every time we do the transformation to SQL, we re-parse it. What really takes time is validation (use metadata like getting it from HMS) and optimization. Regards! Aron Tao Juan Pan <panj...@apache.org> 于2020年11月22日周日 下午2:32写道: Hi community, Thanks for your attention. : ) Currently, Apache ShardingSphere community plans to leverage Apache Calcite to implement federated SQL query, i.e., the query from different database instances [1]. The draft approach is that we consider using the custom adaptor with the SQL parser of ShardingSphere itself (Antlr involved), and transforming the parsed result to the algebra of Calcite. Lastly, Calcite will execute the SQLs by means of the custom adaptor. Currently, I know the entrance of calling the custom adaptor is to use the `DriverManager.getConnection(CalciteUrl)`, which will get Calcite's SQL parsing involved. But we want to avoid twice SQL parsing, which means we wish to ignore the SQL parsing of CalciteN . My question is that how we can leverage Calcite adaptor without using Calcite parser. Could you give me some hints? Very appreciated your any help and reply. Regards, Trista [1] https://github.com/apache/shardingsphere/issues/8284 Juan Pan (Trista) Senior DBA & PMC of Apache ShardingSphere E-mail: panj...@apache.org