adriangbot commented on PR #10554: URL: https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404770972
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404308859) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) Comparing claude/parquet-exact-value-windows-10538 (c7591d17b58bd3db5cd2201a272653837da9c8c6) to 3577093f26591aee806a9133c891b6b8e5255477 [diff](https://github.com/apache/arrow-rs/compare/3577093f26591aee806a9133c891b6b8e5255477..c7591d17b58bd3db5cd2201a272653837da9c8c6) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: "^[m-z]" baseline: ref: "3577093f26591aee806a9133c891b6b8e5255477" ``` </details> <details><summary>CPU Details (lscpu)</summary> ``` Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping: r0p1 BogoMIPS: 2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache: 32 MiB (16 instances) L3 cache: 80 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Details</summary> <p> ``` group claude_parquet-exact-value-windows-10538 main ----- ---------------------------------------- ---- medium_string_shared_prefix_nullable/delta_byte_array 1.01 40.2±0.24ms 6.2 GB/sec 1.00 39.8±0.27ms 6.3 GB/sec primitive/bloom_filter 1.00 145.4±0.59ms 308.6 MB/sec 1.01 146.8±0.52ms 305.8 MB/sec primitive/cdc 1.00 153.4±0.80ms 292.5 MB/sec 1.00 153.5±0.49ms 292.3 MB/sec primitive/default 1.01 112.6±0.42ms 398.5 MB/sec 1.00 111.8±0.39ms 401.3 MB/sec primitive/number_distinct_values 1.00 361.3±1.10ms 124.2 MB/sec 1.00 361.0±1.01ms 124.3 MB/sec primitive/parquet_2 1.01 127.4±0.38ms 352.1 MB/sec 1.00 126.8±0.42ms 354.0 MB/sec primitive/zstd 1.01 142.4±0.41ms 315.1 MB/sec 1.00 141.4±0.36ms 317.3 MB/sec primitive/zstd_parquet_2 1.01 160.5±0.45ms 279.6 MB/sec 1.00 159.7±0.35ms 281.0 MB/sec primitive_all_null/bloom_filter 1.01 919.3±5.85µs 47.7 GB/sec 1.00 912.1±10.44µs 48.0 GB/sec primitive_all_null/cdc 1.00 18.3±0.40ms 2.4 GB/sec 1.03 19.0±0.36ms 2.3 GB/sec primitive_all_null/default 1.00 276.0±1.47µs 158.8 GB/sec 1.00 276.6±2.06µs 158.5 GB/sec primitive_all_null/number_distinct_values 1.00 277.4±1.18µs 158.0 GB/sec 1.00 278.1±2.11µs 157.6 GB/sec primitive_all_null/parquet_2 1.00 284.3±1.71µs 154.1 GB/sec 1.00 284.0±2.01µs 154.3 GB/sec primitive_all_null/zstd 1.00 424.7±1.16µs 103.2 GB/sec 1.01 429.6±2.02µs 102.0 GB/sec primitive_all_null/zstd_parquet_2 1.00 379.5±0.94µs 115.5 GB/sec 1.01 382.0±2.33µs 114.7 GB/sec primitive_non_null/bloom_filter 1.00 100.7±0.59ms 437.0 MB/sec 1.03 103.8±0.58ms 424.1 MB/sec primitive_non_null/cdc 1.00 83.1±0.50ms 529.8 MB/sec 1.00 82.9±0.34ms 531.0 MB/sec primitive_non_null/default 1.00 60.1±0.35ms 732.2 MB/sec 1.01 60.6±0.38ms 726.3 MB/sec primitive_non_null/number_distinct_values 1.00 479.8±2.20ms 91.7 MB/sec 1.01 482.5±2.29ms 91.2 MB/sec primitive_non_null/parquet_2 1.00 82.1±0.33ms 536.0 MB/sec 1.00 82.4±0.43ms 534.2 MB/sec primitive_non_null/zstd 1.01 91.9±0.48ms 478.8 MB/sec 1.00 91.1±0.38ms 483.1 MB/sec primitive_non_null/zstd_parquet_2 1.00 115.5±0.50ms 380.9 MB/sec 1.00 115.5±0.35ms 381.0 MB/sec primitive_sparse_99pct_null/bloom_filter 1.08 13.3±0.48ms 3.3 GB/sec 1.00 12.3±0.32ms 3.6 GB/sec primitive_sparse_99pct_null/cdc 1.00 30.1±0.34ms 1490.0 MB/sec 1.08 32.4±0.24ms 1383.0 MB/sec primitive_sparse_99pct_null/default 1.04 11.1±0.06ms 4.0 GB/sec 1.00 10.6±0.04ms 4.1 GB/sec primitive_sparse_99pct_null/number_distinct_values 1.00 15.0±0.08ms 2.9 GB/sec 1.01 15.1±0.13ms 2.9 GB/sec primitive_sparse_99pct_null/parquet_2 1.03 11.1±0.06ms 4.0 GB/sec 1.00 10.8±0.09ms 4.1 GB/sec primitive_sparse_99pct_null/zstd 1.02 14.5±0.05ms 3.0 GB/sec 1.00 14.3±0.20ms 3.1 GB/sec primitive_sparse_99pct_null/zstd_parquet_2 1.04 13.3±0.09ms 3.3 GB/sec 1.00 12.8±0.14ms 3.4 GB/sec short_string_non_null/bloom_filter 1.05 28.1±0.07ms 427.4 MB/sec 1.00 26.9±0.10ms 446.9 MB/sec short_string_non_null/cdc 1.00 20.4±0.04ms 587.9 MB/sec 1.00 20.4±0.06ms 587.1 MB/sec short_string_non_null/default 1.01 16.3±0.04ms 738.3 MB/sec 1.00 16.2±0.06ms 742.6 MB/sec short_string_non_null/number_distinct_values 1.02 69.5±0.26ms 172.6 MB/sec 1.00 68.4±0.17ms 175.5 MB/sec short_string_non_null/parquet_2 1.00 25.9±0.03ms 463.0 MB/sec 1.00 25.9±0.11ms 463.5 MB/sec short_string_non_null/zstd 1.00 36.5±0.09ms 329.1 MB/sec 1.00 36.5±0.10ms 329.2 MB/sec short_string_non_null/zstd_parquet_2 1.01 28.9±0.06ms 415.3 MB/sec 1.00 28.7±0.08ms 417.5 MB/sec small_string_distinct/delta_byte_array 1.06 1066.8±14.63µs 7.4 GB/sec 1.00 1007.7±13.76µs 7.8 GB/sec small_string_distinct/plain 1.08 814.1±6.98µs 9.6 GB/sec 1.00 755.1±6.34µs 10.4 GB/sec small_string_partial_prefix/delta_byte_array 1.10 1100.3±12.67µs 7.1 GB/sec 1.00 1000.1±6.58µs 7.8 GB/sec small_string_partial_prefix/plain 1.09 1080.5±10.20µs 7.3 GB/sec 1.00 993.0±6.80µs 7.9 GB/sec small_string_shared_prefix/delta_byte_array 1.05 1017.3±5.46µs 7.7 GB/sec 1.00 966.3±2.35µs 8.1 GB/sec small_string_shared_prefix/plain 1.04 1163.1±10.36µs 6.7 GB/sec 1.00 1122.2±5.81µs 7.0 GB/sec string/bloom_filter 1.07 231.5±26.24ms 2.2 GB/sec 1.00 216.7±22.53ms 2.4 GB/sec string/cdc 1.00 209.3±9.11ms 2.4 GB/sec 1.00 209.7±10.66ms 2.4 GB/sec string/default 1.00 112.7±12.41ms 4.5 GB/sec 1.21 136.3±25.25ms 3.8 GB/sec string/number_distinct_values 1.00 232.4±9.82ms 2.2 GB/sec 1.02 238.2±22.29ms 2.1 GB/sec string/parquet_2 1.00 184.2±0.57ms 2.8 GB/sec 1.03 190.2±0.74ms 2.7 GB/sec string/zstd 1.00 449.9±17.20ms 1165.4 MB/sec 1.05 470.5±18.14ms 1114.3 MB/sec string/zstd_parquet_2 1.00 412.5±20.78ms 1270.9 MB/sec 1.09 448.3±1.74ms 1169.4 MB/sec string_and_binary_view/bloom_filter 1.01 67.5±0.36ms 477.4 MB/sec 1.00 66.6±0.19ms 484.2 MB/sec string_and_binary_view/cdc 1.01 61.7±0.19ms 523.1 MB/sec 1.00 61.2±0.12ms 526.9 MB/sec string_and_binary_view/default 1.00 50.6±0.16ms 637.5 MB/sec 1.00 50.4±0.11ms 639.9 MB/sec string_and_binary_view/number_distinct_values 1.00 50.5±0.16ms 638.4 MB/sec 1.00 50.4±0.11ms 640.2 MB/sec string_and_binary_view/parquet_2 1.00 61.5±0.15ms 524.8 MB/sec 1.00 61.3±0.09ms 526.2 MB/sec string_and_binary_view/zstd 1.00 87.6±0.17ms 368.1 MB/sec 1.00 87.6±0.11ms 368.1 MB/sec string_and_binary_view/zstd_parquet_2 1.01 75.6±0.23ms 426.4 MB/sec 1.00 75.1±0.11ms 429.2 MB/sec string_dictionary/bloom_filter 1.00 88.9±0.38ms 2.9 GB/sec 1.21 107.2±1.29ms 2.4 GB/sec string_dictionary/cdc 1.00 51.7±1.74ms 5.0 GB/sec 1.30 67.3±0.83ms 3.8 GB/sec string_dictionary/default 1.00 44.6±0.27ms 5.8 GB/sec 1.41 63.0±0.32ms 4.1 GB/sec string_dictionary/number_distinct_values 1.00 77.4±0.71ms 3.3 GB/sec 1.24 96.3±0.46ms 2.7 GB/sec string_dictionary/parquet_2 1.00 48.5±0.49ms 5.3 GB/sec 1.33 64.3±0.16ms 4.0 GB/sec string_dictionary/zstd 1.00 211.7±3.98ms 1247.6 MB/sec 1.05 222.1±0.80ms 1189.1 MB/sec string_dictionary/zstd_parquet_2 1.00 192.7±0.24ms 1370.5 MB/sec 1.05 202.0±5.40ms 1307.4 MB/sec string_dictionary_low_cardinality_100/bloom_filter 1.00 33.3±0.05ms 120.4 MB/sec 1.01 33.7±0.19ms 119.0 MB/sec string_dictionary_low_cardinality_100/cdc 1.00 26.8±0.07ms 149.5 MB/sec 1.01 27.0±0.21ms 148.1 MB/sec string_dictionary_low_cardinality_100/default 1.00 24.3±0.05ms 164.7 MB/sec 1.01 24.6±0.14ms 162.6 MB/sec string_dictionary_low_cardinality_100/number_distinct_values 1.00 40.3±0.13ms 99.5 MB/sec 1.00 40.3±0.20ms 99.3 MB/sec string_dictionary_low_cardinality_100/parquet_2 1.00 24.3±0.04ms 164.5 MB/sec 1.02 24.8±0.16ms 161.5 MB/sec string_dictionary_low_cardinality_100/zstd 1.00 24.7±0.05ms 162.2 MB/sec 1.01 25.1±0.16ms 159.8 MB/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.00 24.7±0.06ms 162.1 MB/sec 1.01 25.1±0.15ms 159.9 MB/sec string_dictionary_low_cardinality_20/bloom_filter 1.00 32.7±0.06ms 122.4 MB/sec 1.01 33.1±0.18ms 121.0 MB/sec string_dictionary_low_cardinality_20/cdc 1.00 27.2±0.10ms 147.4 MB/sec 1.00 27.1±0.23ms 147.9 MB/sec string_dictionary_low_cardinality_20/default 1.00 24.6±0.05ms 163.0 MB/sec 1.00 24.6±0.16ms 162.6 MB/sec string_dictionary_low_cardinality_20/number_distinct_values 1.00 40.2±0.18ms 99.6 MB/sec 1.00 40.1±0.28ms 99.8 MB/sec string_dictionary_low_cardinality_20/parquet_2 1.00 24.6±0.05ms 162.9 MB/sec 1.00 24.6±0.16ms 162.5 MB/sec string_dictionary_low_cardinality_20/zstd 1.00 24.7±0.05ms 162.1 MB/sec 1.00 24.8±0.17ms 161.5 MB/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.00 24.8±0.05ms 161.8 MB/sec 1.00 24.8±0.16ms 161.6 MB/sec string_dictionary_low_cardinality_400/bloom_filter 1.00 34.4±0.07ms 116.7 MB/sec 1.00 34.5±0.17ms 116.3 MB/sec string_dictionary_low_cardinality_400/cdc 1.00 27.4±0.07ms 146.2 MB/sec 1.00 27.3±0.15ms 146.8 MB/sec string_dictionary_low_cardinality_400/default 1.00 25.0±0.07ms 160.3 MB/sec 1.00 25.1±0.17ms 159.6 MB/sec string_dictionary_low_cardinality_400/number_distinct_values 1.00 40.8±0.10ms 98.2 MB/sec 1.00 40.9±0.18ms 98.1 MB/sec string_dictionary_low_cardinality_400/parquet_2 1.00 25.1±0.07ms 160.1 MB/sec 1.01 25.2±0.16ms 159.2 MB/sec string_dictionary_low_cardinality_400/zstd 1.00 25.6±0.06ms 157.0 MB/sec 1.01 25.7±0.18ms 156.1 MB/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.00 25.5±0.06ms 157.1 MB/sec 1.00 25.7±0.16ms 156.4 MB/sec string_non_null/bloom_filter 1.00 252.3±17.36ms 2.0 GB/sec 1.08 273.1±29.05ms 1918.8 MB/sec string_non_null/cdc 1.01 249.9±5.37ms 2.0 GB/sec 1.00 248.3±5.82ms 2.1 GB/sec string_non_null/default 1.00 128.6±19.91ms 4.0 GB/sec 1.14 146.6±26.86ms 3.5 GB/sec string_non_null/number_distinct_values 1.02 289.4±5.53ms 1810.3 MB/sec 1.00 282.7±4.82ms 1853.8 MB/sec string_non_null/parquet_2 1.00 131.3±10.10ms 3.9 GB/sec 1.03 135.9±0.73ms 3.8 GB/sec string_non_null/zstd 1.00 570.2±14.11ms 919.0 MB/sec 1.00 570.7±15.58ms 918.1 MB/sec string_non_null/zstd_parquet_2 1.00 491.9±8.14ms 1065.3 MB/sec 1.01 496.2±6.74ms 1056.1 MB/sec string_ree/bloom_filter 1.00 108.6±2.38ms 151.1 MB/sec 1.45 157.6±1.07ms 104.1 MB/sec string_ree/cdc 1.00 177.9±0.63ms 92.3 MB/sec 1.26 223.4±1.88ms 73.4 MB/sec string_ree/default 1.00 81.6±1.10ms 201.0 MB/sec 1.59 129.5±1.08ms 126.7 MB/sec string_ree/number_distinct_values 1.00 119.2±0.38ms 137.7 MB/sec 1.37 162.7±2.04ms 100.8 MB/sec string_ree/parquet_2 1.00 80.6±0.35ms 203.5 MB/sec 1.61 130.1±0.33ms 126.1 MB/sec string_ree/zstd 1.00 141.6±0.65ms 115.9 MB/sec 1.34 189.7±0.87ms 86.5 MB/sec string_ree/zstd_parquet_2 1.00 95.0±0.31ms 172.7 MB/sec 1.53 145.0±0.37ms 113.2 MB/sec string_ree_95pct_null/bloom_filter 1.02 26.7±0.10ms 52.6 MB/sec 1.00 26.2±0.12ms 53.8 MB/sec string_ree_95pct_null/cdc 1.01 32.8±0.08ms 42.9 MB/sec 1.00 32.3±0.14ms 43.5 MB/sec string_ree_95pct_null/default 1.01 25.0±0.04ms 56.2 MB/sec 1.00 24.7±0.08ms 57.0 MB/sec string_ree_95pct_null/number_distinct_values 1.01 27.2±0.04ms 51.7 MB/sec 1.00 26.9±0.09ms 52.4 MB/sec string_ree_95pct_null/parquet_2 1.02 25.1±0.04ms 56.1 MB/sec 1.00 24.7±0.08ms 57.1 MB/sec string_ree_95pct_null/zstd 1.02 25.9±0.05ms 54.3 MB/sec 1.00 25.5±0.09ms 55.2 MB/sec string_ree_95pct_null/zstd_parquet_2 1.01 25.8±0.04ms 54.5 MB/sec 1.00 25.6±0.22ms 55.0 MB/sec struct_all_null/bloom_filter 1.01 380.8±2.72µs 41.4 GB/sec 1.00 377.4±1.42µs 41.7 GB/sec struct_all_null/cdc 1.00 7.3±0.09ms 2.2 GB/sec 1.22 8.9±0.26ms 1806.8 MB/sec struct_all_null/default 1.00 117.9±0.52µs 133.6 GB/sec 1.01 119.2±0.92µs 132.1 GB/sec struct_all_null/number_distinct_values 1.00 119.2±0.57µs 132.1 GB/sec 1.00 119.3±0.92µs 132.0 GB/sec struct_all_null/parquet_2 1.00 121.1±0.55µs 130.1 GB/sec 1.01 121.8±0.85µs 129.3 GB/sec struct_all_null/zstd 1.00 182.4±0.50µs 86.4 GB/sec 1.00 182.6±0.83µs 86.2 GB/sec struct_all_null/zstd_parquet_2 1.01 164.4±0.84µs 95.8 GB/sec 1.00 162.9±0.97µs 96.7 GB/sec struct_non_null/bloom_filter 1.00 43.1±0.18ms 371.4 MB/sec 1.01 43.4±0.25ms 368.6 MB/sec struct_non_null/cdc 1.00 42.1±0.24ms 380.1 MB/sec 1.00 42.2±0.26ms 379.3 MB/sec struct_non_null/default 1.00 28.6±0.17ms 560.3 MB/sec 1.00 28.5±0.19ms 561.4 MB/sec struct_non_null/number_distinct_values 1.03 192.5±0.92ms 83.1 MB/sec 1.00 186.2±0.65ms 85.9 MB/sec struct_non_null/parquet_2 1.00 37.3±0.21ms 429.2 MB/sec 1.00 37.3±0.19ms 428.8 MB/sec struct_non_null/zstd 1.01 37.4±0.24ms 427.9 MB/sec 1.00 37.2±0.22ms 430.5 MB/sec struct_non_null/zstd_parquet_2 1.00 51.0±0.23ms 313.7 MB/sec 1.00 51.1±0.19ms 313.0 MB/sec struct_sparse_99pct_null/bloom_filter 1.00 6.9±0.09ms 2.3 GB/sec 1.00 6.9±0.08ms 2.3 GB/sec struct_sparse_99pct_null/cdc 1.00 13.8±0.11ms 1168.3 MB/sec 1.09 15.1±0.20ms 1070.0 MB/sec struct_sparse_99pct_null/default 1.00 6.2±0.05ms 2.5 GB/sec 1.02 6.3±0.03ms 2.5 GB/sec struct_sparse_99pct_null/number_distinct_values 1.02 7.9±0.08ms 2035.9 MB/sec 1.00 7.8±0.07ms 2.0 GB/sec struct_sparse_99pct_null/parquet_2 1.00 6.2±0.09ms 2.6 GB/sec 1.03 6.4±0.07ms 2.5 GB/sec struct_sparse_99pct_null/zstd 1.00 7.6±0.05ms 2.1 GB/sec 1.02 7.7±0.04ms 2.0 GB/sec struct_sparse_99pct_null/zstd_parquet_2 1.02 7.2±0.09ms 2.2 GB/sec 1.00 7.1±0.04ms 2.2 GB/sec ``` </p> </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1910.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1764.8s | | CPU sys | 139.3s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1865.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1800.9s | | CPU sys | 61.4s | | Peak spill | 0 B | </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
