wjones127 commented on PR #13857: URL: https://github.com/apache/arrow/pull/13857#issuecomment-1251563275
> The numbers seem a bit low to be honest You are correct on that. Both too low in test and baseline, by about the same factor. I was creating too large of a chunked array for the indices. <details><summary> Benchmark results </summary> Baseline: ``` -------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------------------------------------- TakeChunkedInt64RandomIndicesNoNulls/4194304/1000 2770528 ns 2769802 ns 232 items_per_second=1.5143G/s null_percent=0.1 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/10 2802730 ns 2802061 ns 246 items_per_second=1.49686G/s null_percent=10 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/2 4277424 ns 4276390 ns 164 items_per_second=980.805M/s null_percent=50 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/1 2143412 ns 2142790 ns 305 items_per_second=1.9574G/s null_percent=100 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/0 1886756 ns 1886230 ns 374 items_per_second=2.22364G/s null_percent=0 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/1000 2799584 ns 2799301 ns 249 items_per_second=1.49834G/s null_percent=0.1 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/10 3865247 ns 3864123 ns 179 items_per_second=1085.45M/s null_percent=10 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/2 6030057 ns 6028330 ns 103 items_per_second=695.765M/s null_percent=50 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/1 1441460 ns 1440831 ns 498 items_per_second=2.91103G/s null_percent=100 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/0 1923944 ns 1923507 ns 371 items_per_second=2.18055G/s null_percent=0 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/1000 1827538 ns 1827005 ns 383 items_per_second=2.29573G/s null_percent=0.1 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/10 1806226 ns 1805824 ns 387 items_per_second=2.32265G/s null_percent=10 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/2 2600097 ns 2599818 ns 269 items_per_second=1.61331G/s null_percent=50 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/1 1069649 ns 1069057 ns 667 items_per_second=3.92337G/s null_percent=100 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/0 949020 ns 948810 ns 738 items_per_second=4.42059G/s null_percent=0 size=4.1943M ``` Proposed: ``` -------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------------------------------------- TakeChunkedInt64RandomIndicesNoNulls/4194304/1000 12734298 ns 12731491 ns 53 items_per_second=329.443M/s null_percent=0.1 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/10 13030870 ns 13027741 ns 54 items_per_second=321.952M/s null_percent=10 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/2 11699983 ns 11697067 ns 60 items_per_second=358.577M/s null_percent=50 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/1 8201179 ns 8200176 ns 85 items_per_second=511.489M/s null_percent=100 size=4.1943M TakeChunkedInt64RandomIndicesNoNulls/4194304/0 8281752 ns 8280094 ns 85 items_per_second=506.553M/s null_percent=0 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/1000 10054480 ns 10052444 ns 54 items_per_second=417.242M/s null_percent=0.1 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/10 10256613 ns 10254956 ns 68 items_per_second=409.003M/s null_percent=10 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/2 8735435 ns 8734213 ns 80 items_per_second=480.215M/s null_percent=50 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/1 98487 ns 98479 ns 7191 items_per_second=42.5911G/s null_percent=100 size=4.1943M TakeChunkedInt64RandomIndicesWithNulls/4194304/0 8187746 ns 8186131 ns 84 items_per_second=512.367M/s null_percent=0 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/1000 2008840 ns 2007886 ns 352 items_per_second=2.08892G/s null_percent=0.1 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/10 2467236 ns 2466807 ns 285 items_per_second=1.7003G/s null_percent=10 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/2 3366623 ns 3365923 ns 208 items_per_second=1.24611G/s null_percent=50 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/1 1295032 ns 1294838 ns 538 items_per_second=3.23925G/s null_percent=100 size=4.1943M TakeChunkedInt64MonotonicIndices/4194304/0 1240969 ns 1240848 ns 566 items_per_second=3.38019G/s null_percent=0 size=4.1943M ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
