Thanks for taking a look! > On 27 Apr 2021, at 19:25, Rui Wang <[email protected]> wrote: > > Not every query can be supported by BeamSQL easily. For example, support non > equi-join(BEAM-2194).
Could you point me out why "non equi-join” can’t be supported? Either it can and this is just a question of implementation? > We had discussions for cause 2 to add the limitation that BeamSQL only > supports ORDER BY LIMIT (LIMIT is required). I’m curious what is a current implementation of "ORDER BY LIMIT” and can it be applied, at least, to only Bounded collection/Global window in the same way for "ORDER BY" without limits? > Cause 3 needs a case by case investigation, some might be able to be fixed. > Cause 4 looks like no such function found in the catalog. I guess it should be "SUBSTRING(<CHARACTER> FROM <NUMERIC> FOR <NUMERIC>)” instead of "substr(<CHARACTER>, <NUMERIC>, <NUMERIC>)” ? — Alexey > > I’m not very familiar with a current status of ongoing work for Beam SQL, so > I’m sorry in advance if my questions will sound naive. > > Please, guide me on this: > > 1. Are there any chances that we can resolve, at least, partly the current > limitations of the query parsing/planning, mentioned above? Are there any > principal blockers among them? > 2. Are there any plans or ongoing work related to this? > 3. Are there any plans to upgrade vendored Calcite version to more recent > one? Should it reduce the number of current limitations or not? > 4. Do you think it could be valuable for Beam SQL to run TPC-DS benchmark on > a regular basis (as we do for Nexmark, for example) even if not all queries > can pass with Beam SQL? > > This is definitely valuable for BeamSQL if we have enough resources to run > such queries regularly. > > I’d appreciate any additional information/docs/details/opinions on this topic. > > — > Alexey > > [1] https://github.com/apache/beam/tree/master/sdks/java/testing/tpcds > <https://github.com/apache/beam/tree/master/sdks/java/testing/tpcds> > [2] http://www.tpc.org/tpcds/ <http://www.tpc.org/tpcds/> > [3] > https://docs.google.com/spreadsheets/d/1Gya9Xoa6uWwORHSrRqpkfSII4ajYvDpUTt0cNJCRHjE/edit?usp=sharing > > <https://docs.google.com/spreadsheets/d/1Gya9Xoa6uWwORHSrRqpkfSII4ajYvDpUTt0cNJCRHjE/edit?usp=sharing> > [4] > https://github.com/apache/beam/tree/master/sdks/java/testing/tpcds/src/main/resources/queries > > <https://github.com/apache/beam/tree/master/sdks/java/testing/tpcds/src/main/resources/queries>
