Hello, A MariaDB query optimizer member here.
I think it would be nice to get VIDEX for MariaDB. As far I'm informed, the basics of interaction between the query optimizer and storage engine are the same between MariaDB and MySQL: - ha_engine::info() call - ha_engine::records_in_range() call - extended keys feature Execution part (index/table read methods) is also very similar although there are some cosmetic differences. There are bigger differences in how partitioning is handled, data dictionary and column histograms. I think it's feasible to work around those. To sum up, at first glance adoption to MariaDB seems doable. I'd like to study VINDEX further and see where/how we MariaDB devs could lend a hand. Best Regards, -- Sergei Petrunia Query Optimizer team MariaDB Corporation On Wed, Apr 16, 2025 at 09:41:16PM -0500, 康荣 via discuss wrote: > Dear MariaDB Community, > > I hope this message finds you well. We'd like to introduce VIDEX (Virtual > Index > for MySQL) and humbly seek your guidance and assistance in adapting it for > MariaDB compatibility. > > We've developed a virtual index storage engine (plugin) called VIDEX. It can > simulate the impact of indexes on query plan costs without requiring real > data. > VIDEX forwards the statistical information, ndv (records_per_key), and > cardinality information (records_in_range) needed for making query plans via a > RESTful approach, with responses provided by an "algorithm service." This > disaggregated design is particularly well-suited for integrating various > AI-boosted cardinality and ndv algorithms. VIDEX has been successfully > deployed > at ByteDance, serving thousands of MySQL instances daily and over millions of > SQL queries for index optimization tasks. > > We implemented VIDEX based on MySQL 8.0 and now, thanks to Federico Razzoli > for > his suggestion (LinkedIn post 1, LinkedIn post 2, , Github Issue), we'd like > to > adapt it to MariaDB. However, we've discovered significant differences between > MySQL 8.0 and the latest MariaDB codebase. Therefore, we're seeking help from > the MariaDB community, and would greatly appreciate collaboration with > experienced developers to modify the code. We're very willing to cooperate in > this effort. In the future, we also hope to have the opportunity to add VIDEX > to the official MariaDB plugin list. > > About VIDEX > VIDEX GitHub repository: https://github.com/bytedance/videx/ > > Virtual index technology is already available in PostgreSQL, Oracle, and IBM > DB2, and would provide MariaDB users with valuable capabilities for query > optimization and database research. > VIDEX offers key features that would benefit MariaDB users: > > • What-if analysis for index strategies without production impact > • Support for integration of custom or AI-based estimation algorithms > • Accurate simulation of query plans, verified with complex benchmarks like > TPC-H and JOB > > > If you're interested in collaborating or have guidance to offer on adapting > our > code to MariaDB, we'd greatly appreciate hearing from you. > > Thank you for your consideration. > > Best regards, > Rong > ByteBrain Team, Bytedance > _______________________________________________ > discuss mailing list -- [email protected] > To unsubscribe send an email to [email protected] _______________________________________________ discuss mailing list -- [email protected] To unsubscribe send an email to [email protected]
