sam-1112 commented on PR #5415: URL: https://github.com/apache/datafusion-comet/pull/5415#issuecomment-5462229301
Thanks — you were right that the aggregate expansion and group-depth limits still missed compilation cost from deeply nested quantifiers. I addressed this in two separate commits: 1. [`63c938a3`](https://github.com/apache/datafusion-comet/pull/5415/commits/63c938a3e1a251725098c4e279fa8801fde944db) adds `MaxQuantifierNesting = 8`. The reported 105-byte pattern is now classified as `Incompatible` and remains on the JVM dispatcher. The accounting takes the maximum nesting across concatenated siblings rather than summing them, so ordinary sibling quantifiers such as repeated `a*` are unaffected. 2. While validating the remaining boundary against the locked `regex` 1.13.1, I found another admitted `CompiledTooBig` shape: seven nested capturing `*` groups over `[^;]`, 16 additional capturing wrappers, and sixteen concatenated `{256}` copies. [`da20a2c4`](https://github.com/apache/datafusion-comet/pull/5415/commits/da20a2c41da901634addd6b553bbbf3fde7367bc) accounts for the `CaptureStart` / `CaptureEnd` states added by each capturing group before counted repetition multiplies the inner expression. Non-capturing groups do not add this cost, and `*`, `+`, and `?` do not multiply it. Coverage now includes: - analyzer and dispatcher-routing regressions for your 105-byte nested-quantifier repro; - analyzer and dispatcher-routing regressions for the capture-cost residual; - the exact `MaxQuantifierNesting` boundary and concatenated-sibling control; - continued native admission for `[^x]{256}`, `[^;]{256}` repeated 16 times, `(?:a{64}){64}`, `(?:foo){256}`, and 32 nested groups without counted capture duplication; - rejection of the capturing `(a{64}){64}` counterpart because counted repetition copies its capture states. I did not add a compile-then-fallback path. These limits remain conservative structural heuristics rather than a formal compiled-byte guarantee, but both concrete admitted `CompiledTooBig` patterns verified against the locked 1.13.1 engine now stay off automatic native routing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
