markhoerth commented on PR #14332: URL: https://github.com/apache/gravitino/pull/14332#issuecomment-6696489996
Thank you @dengc367 and @charangowdamd-cmd. The problem in #14308 is real. When every consumer resolves a semantic model on its own, numbers drift, and resolution belongs in Gravitino where it can be governed once. Apache Ossie is now defining exactly these semantics in [PR #246, "Foundational Semantics"](https://github.com/apache/ossie/pull/246). Under it, the caller names dimensions and measures, and the engine chooses the join path and join types. Facts join LEFT so no row is dropped, and separate fact tables are combined with a FULL OUTER join. Ambiguous paths and fan-out are refused with typed errors rather than answered. Whatever Gravitino adopts for semantic queries should conform to it, so that Gravitino gives the same answer as any other Ossie engine. Against that bar, the current PR has three gaps: 1. **The agent receives a plan rather than something it can run.** `plan_semantic_query` returns a plan, and the PR includes no converter, so the agent must write the SQL, including splicing in each metric's expression from a raw SQL string. The final number then depends on agent-written SQL, which is the drift #14308 sets out to remove. 2. **The resolver doesn't meet PR #246's guarantees.** - It always joins INNER, which drops fact rows with no match (PR #246 Semantic 1). - It takes the shortest path where PR #246 requires refusal with `E_AMBIGUOUS_PATH` (ยง6.9). - It has no fan-out handling (Semantic 2). 3. **The plan format is Gravitino-specific.** If Gravitino exposes a plan, Substrait is the existing standard for relational plans. A bespoke format would be one the project alone defines, maintains and has to get adopted. One smaller point: the docs say a caller without access to a referenced source receives a 403. The `/plan` endpoint, however, checks only access to the semantic model. We have a semantic query compiler effort in progress, and we're evaluating how it conforms to PR #246. I'd suggest holding this PR until that direction is clearer, so the project doesn't take on two approaches to the same problem. @dengc367, feedback from your experience running a planner against real agent workloads would be welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
