Dear all, My colleagues and I are completing the implementation of the first benchmark suite for Calcite, with the expectation to have something in place in the next 2-4 weeks, and available for testing.
The plan is to make it available for your review and testing, and then release it Current status is: We have 80+ out of 99 TPC-DS queries made compatible to Calcite for Postgres backend. (Calcite-178) It will be compatible with most of the backends, but the queries will need testing and validation. We need to complete following two tasks to complete the initial benchmark work: 1) an accurate method to measure and instrument the Calcite performance since the time measurement for Calcite may involve overhead associated with Avatica/JDBC. 2) fix queries that require syntax modification for Calcite syntax or to account for some lack of support We are not sure yet how bad the 2 part might be. It might involve some bug patches or workarounds, which also might vary from backend to backend. Big thanks to my R&D team members, Dr. Seung-Hwan Lim and Ashwin Vajantri for working on this, and specially on tedious TPC-DS query syntax fixing to make it work with Calcite.
