LucaCappelletti94 commented on code in PR #2352:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/2352#discussion_r3313355980
##########
src/parser/mod.rs:
##########
@@ -1717,6 +1731,23 @@ impl<'a> Parser<'a> {
return prefix;
}
+ // Memoize parse_prefix failures to break 2^N speculation when both
+ // prefix arms fail at every level (e.g. `IF(current_time(...x`).
+ // The per-arm cache in `parse_prefix_inner` complements this for
+ // chains where the reserved arm fails but the unreserved fallback
+ // succeeds (e.g. `case-case-...c`).
+ let start_index = self.index;
+ if let Some(cached) = self.failed_prefix_positions.get(&start_index) {
+ return Err(cached.clone());
+ }
+ let result = self.parse_prefix_inner();
+ if let Err(ref e) = result {
+ self.failed_prefix_positions.insert(start_index, e.clone());
Review Comment:
<img width="2380" height="756" alt="image"
src="https://github.com/user-attachments/assets/802297f2-e2a9-496d-9a86-a56d23fb169a";
/>
Got a scaling for the cache memory. The two caches are keyed by token index,
so each entry is about 40 bytes (8 byte usize key plus a 32 byte ParserError).
Panels plot entries, heap String bytes, and total memory vs chain length N for
valid SQL, the nested if(current_time(...x chain, and the wide case<TAB>-...c
chain.
- On valid SQL both maps stay empty (0 entries at a 27 KiB SELECT).
- Heap String bytes are bounded by recursion depth, not input: past the
recursion limit the cached error is RecursionLimitExceeded with no String, so
strings peak at ~1.8 KiB and never grow with input.
- Total memory is linear with a small constant: capped at ~4 KiB for deep
nesting, at most ~13x input for the adversarial wide shape (159 KiB at a 12 KiB
input). Reaching 2 GB would need a ~154 MB single statement.
So it is linear and never exponential, and valid SQL costs nothing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
