Abhisheklearn12 commented on PR #10436: URL: https://github.com/apache/arrow-rs/pull/10436#issuecomment-5153002380
also i re-ran after the split, since removing the checks touched `view_from_dict_values` on the hot path. three runs, max spread 0.9 points. **existing fast paths on main** | cast | rows | dict values | main | this PR | change | |---|---|---|---|---|---| | `Utf8->Utf8View` | 8,192 | 100 | 25.8 us | 3.4 us | **-86.9%** | | `Utf8->Utf8View` | 1,000,000 | 1,000 | 3207.1 us | 543.0 us | **-83.1%** | | `Binary->BinaryView` | 8,192 | 100 | 23.7 us | 3.4 us | **-85.8%** | | `Binary->BinaryView` | 1,000,000 | 1,000 | 3206.9 us | 544.1 us | **-83.0%** | **new arms, dictionary larger than the array** | cast | rows | dict values | main | this PR | change | |---|---|---|---|---|---| | `LargeUtf8->Utf8View` | 1,000 | 100,000 | 271.7 us | 3.8 us | **-98.6%** | | `LargeUtf8->BinaryView` | 10,000 | 1,000,000 | 15428.6 us | 78.2 us | **-99.5%** | | `LargeBinary->BinaryView` | 1,000 | 100,000 | 273.1 us | 3.9 us | **-98.6%** | | `Utf8->BinaryView` | 10,000 | 1,000,000 | 2867.6 us | 73.8 us | **-97.4%** | | `Binary->Utf8View` | 1,000 | 100,000 | 350.1 us | 87.3 us | **-75.1%** | | `LargeBinary->Utf8View` | 10,000 | 1,000,000 | 4487.2 us | 1652.2 us | **-63.2%** | `Binary`/`LargeBinary -> Utf8View` gain less because UTF-8 validation of the dictionary values dominates. dense shapes on the new arms: -0.4% to +2.5%, the gate routes them to `unpack_dictionary`. both impls compiled into one binary and timed in alternating rounds, outputs asserted equal per cell before timing. i7-11700F. one caveat: my harness's `main` replica calls the direct path straight for the `Utf8->Utf8View` and `Binary->BinaryView` arms, skipping the `cast_with_options` dispatch the real call pays. that is a constant ~0.5us, so those two rows slightly understate the gain. negligible here, but it would dominate on cells of only a few microseconds -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
