apurtell opened a new pull request, #2556: URL: https://github.com/apache/phoenix/pull/2556
Phoenix's `EXPLAIN [WITH REGIONS]` output is today incomplete and easy to misread for both query analysis and performance investigation. Non-trivial joins render with almost no detail. The cost vector advertises three dimensions but only compares IO, and estimates/statistics are unreliable. Some optimizations are incorrectly categorized, for example some shown as row-eliminating predicates are actually column-projection optimizations. The operator trees of complex queries are flattened to a single ident level. Many planner facts are not surfaced at all, such as which rule chooses an index, which indexes the optimizer considered but rejected, what query rewrites took place (e.g. subquery decorrelation, star-join detection, right-to-left normalization, HAVING lift, RVC-offset translation, reverse-scan substitution, UNION ORDER BY, index expression substitution, and more), the specific hash-join strategy chosen, salt bucket counts, local vs. global vs. uncovered-global index distinct ions, the particular flavor of atomic upsert chosen and server-side atomic update expressions, multi tenant context, CDC scope, transaction provider, projection lists, predicate to filter attribution, hints honored vs ignored, and the structure of the JSON/BSON/array path expressions evaluated server-side. This proposal closes all of those gaps. Details provided in the [design document](https://docs.google.com/document/d/10H_MNWGQL7ZzsPmIVBMz7T1jJroez8AYpvwouKcBVQo/edit?tab=t.0) This work was implemented on the dev branch [PHOENIX-7876-feature](https://github.com/apache/phoenix/tree/PHOENIX-7876-feature). All of the individual commits made on the dev branch are preserved as independent commits for this PR. The bulk of the changes are to unit tests and integration tests where they formerly dumped raw EXPLAIN text and did exact string matching against the result. Now every test uses a fluent assertion API against ExplainPlanAttributes. The coverage of the tests has not changed. Their maintainability has improved 1000% (imho). The EXPLAIN features themselves have significantly more test coverage now. Everything I touched I added tests for. Most of them are connectionless tests that don't require a minicluster. [PHOENIX_EXPLAIN_MIGRATION_GUIDE.md](https://issues.apache.org/jira/secure/attachment/13082948/13082948_PHOENIX_EXPLAIN_MIGRATION_GUIDE.md) is a migration guide for anyone scraping today's EXPLAIN PLAN output. [PHOENIX_EXPLAIN_EXAMPLES.md](https://issues.apache.org/jira/secure/attachment/13082944/13082944_PHOENIX_EXPLAIN_EXAMPLES.md) is a companion document for the reviewer's guide that illustrates the new capabilities with ~60 examples that exercise the new and modified EXPLAIN PLAIN features. The following tests are known to have pre-existing failure or flake conditions and flake or fail on current `master`: ```text org.apache.phoenix.end2end.BackwardCompatibilityIT org.apache.phoenix.end2end.GlobalConnectionTenantTableIT org.apache.phoenix.end2end.index.IndexTwoPhaseCreateIT org.apache.phoenix.end2end.index.LocalIndexIT org.apache.phoenix.end2end.IndexScrutinyWithMaxLookbackIT org.apache.phoenix.end2end.transform.TransformIT org.apache.phoenix.end2end.transform.TransformMonitorIT org.apache.phoenix.end2end.transform.TransformToolIT org.apache.phoenix.jdbc.SecureUserConnectionsIT org.apache.phoenix.query.MaxConcurrentConnectionsIT org.apache.phoenix.rpc.UpdateCacheIT ``` I have root caused these failures and have either already filed a JIRA for them or will will shortly. Co-Authored-By: Claude Opus 4.8[1m] <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
