Awesome06 commented on PR #55425: URL: https://github.com/apache/arrow/pull/55425#issuecomment-6845869397
#### What Each Option Means - **OG Baseline (Cast-and-Compare):** Casts `StringView` to standard `String` before comparing (slowest, allocates memory). - **Merged PR #55171 (Eager Iterator):** Direct comparison, but eagerly dereferences variadic buffers on every row via `FromBinaryView`, ignoring the 4-byte inline prefix. - **Option A (This PR #55425 - Direct Prefix):** Checks length and 4-byte prefix in CPU registers before touching variadic buffers; compares inline strings (< 12 B) without `memcmp`. - **Option B (Adaptive Sampling):** Prototyped 32-sample probes to avoid the 14% loss, but discarded because probe overhead slowed down small batches (< 4k rows). --- #### Why Option A is the Right Path Forward - **#55171 intentionally deferred this:** That PR focused on basic kernel coverage. Prefix early-rejection was spun off to #55254 so edge cases, prefix paths, and mismatch distributions could be thoroughly benchmarked without blocking the initial merge. - **Option A beats Option B on predictability:** Option B's 32-sample heuristic avoided the 14% drop on matching strings, but added branch unpredictability and regressed small batches (< 4k rows) by more than 2x. Option A is stateless, deterministic C++. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
