adriangbot commented on PR #10554: URL: https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404823372
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404308267) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) Comparing claude/parquet-exact-value-windows-10538 (c7591d17b58bd3db5cd2201a272653837da9c8c6) to 3577093f26591aee806a9133c891b6b8e5255477 [diff](https://github.com/apache/arrow-rs/compare/3577093f26591aee806a9133c891b6b8e5255477..c7591d17b58bd3db5cd2201a272653837da9c8c6) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: "^[a-l]" baseline: ref: "3577093f26591aee806a9133c891b6b8e5255477" ``` </details> <details><summary>CPU Details (lscpu)</summary> ``` Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping: r0p1 BogoMIPS: 2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache: 32 MiB (16 instances) L3 cache: 80 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Details</summary> <p> ``` group claude_parquet-exact-value-windows-10538 main ----- ---------------------------------------- ---- bool/bloom_filter 1.00 12.7±0.04ms 19.7 MB/sec 1.01 12.8±0.05ms 19.5 MB/sec bool/cdc 1.00 15.8±0.11ms 15.9 MB/sec 1.01 15.8±0.06ms 15.8 MB/sec bool/default 1.00 10.5±0.04ms 23.7 MB/sec 1.01 10.7±0.04ms 23.5 MB/sec bool/number_distinct_values 1.00 21.1±0.05ms 11.9 MB/sec 1.07 22.6±3.30ms 11.0 MB/sec bool/parquet_2 1.00 13.6±0.05ms 18.4 MB/sec 1.00 13.6±0.04ms 18.4 MB/sec bool/zstd 1.00 11.1±0.05ms 22.5 MB/sec 1.01 11.2±0.05ms 22.3 MB/sec bool/zstd_parquet_2 1.00 13.9±0.04ms 18.0 MB/sec 1.00 14.0±0.05ms 17.9 MB/sec bool_non_null/bloom_filter 1.00 6.6±0.03ms 18.8 MB/sec 1.01 6.7±0.03ms 18.6 MB/sec bool_non_null/cdc 1.00 6.5±0.04ms 19.2 MB/sec 1.01 6.6±0.06ms 19.0 MB/sec bool_non_null/default 1.00 3.9±0.02ms 32.5 MB/sec 1.02 3.9±0.02ms 31.9 MB/sec bool_non_null/number_distinct_values 1.00 18.0±0.16ms 7.0 MB/sec 1.11 19.9±4.40ms 6.3 MB/sec bool_non_null/parquet_2 1.00 7.7±0.02ms 16.2 MB/sec 1.00 7.7±0.02ms 16.2 MB/sec bool_non_null/zstd 1.00 4.2±0.02ms 29.7 MB/sec 1.02 4.3±0.02ms 29.3 MB/sec bool_non_null/zstd_parquet_2 1.00 8.1±0.03ms 15.4 MB/sec 1.00 8.1±0.04ms 15.4 MB/sec bool_ree/bloom_filter 1.00 30.2±0.08ms 7.1 MB/sec 1.00 30.2±0.04ms 7.1 MB/sec bool_ree/cdc 1.00 31.8±0.13ms 6.7 MB/sec 1.00 31.7±0.08ms 6.7 MB/sec bool_ree/default 1.00 27.4±0.07ms 7.8 MB/sec 1.00 27.5±0.07ms 7.7 MB/sec bool_ree/number_distinct_values 1.00 40.0±0.06ms 5.3 MB/sec 1.05 41.9±3.91ms 5.1 MB/sec bool_ree/parquet_2 1.00 28.9±0.09ms 7.4 MB/sec 1.00 28.9±0.06ms 7.4 MB/sec bool_ree/zstd 1.00 28.2±0.09ms 7.5 MB/sec 1.00 28.1±0.06ms 7.6 MB/sec bool_ree/zstd_parquet_2 1.00 29.2±0.07ms 7.3 MB/sec 1.00 29.2±0.06ms 7.3 MB/sec decimal/bloom_filter 1.00 167.4±2.66ms 358.4 MB/sec 1.01 168.3±1.38ms 356.5 MB/sec decimal/cdc 1.00 100.5±0.74ms 597.3 MB/sec 1.00 100.7±0.49ms 595.8 MB/sec decimal/default 1.00 124.3±1.54ms 482.7 MB/sec 1.03 127.8±1.28ms 469.5 MB/sec decimal/number_distinct_values 1.00 377.8±4.05ms 158.8 MB/sec 1.03 389.5±16.81ms 154.0 MB/sec decimal/parquet_2 1.00 227.2±2.19ms 264.0 MB/sec 1.01 229.0±3.93ms 262.0 MB/sec decimal/zstd 1.00 138.8±1.23ms 432.2 MB/sec 1.01 140.5±1.18ms 427.0 MB/sec decimal/zstd_parquet_2 1.00 249.5±1.89ms 240.5 MB/sec 1.00 249.6±4.87ms 240.4 MB/sec fixed_size_binary_ree/bloom_filter 1.00 76.2±0.68ms 13.2 MB/sec 1.05 80.1±1.28ms 12.6 MB/sec fixed_size_binary_ree/cdc 1.00 61.7±0.30ms 16.3 MB/sec 1.00 61.9±0.27ms 16.3 MB/sec fixed_size_binary_ree/default 1.00 68.0±0.44ms 14.8 MB/sec 1.06 72.3±1.50ms 13.9 MB/sec fixed_size_binary_ree/number_distinct_values 1.23 87.6±0.35ms 11.5 MB/sec 1.00 71.1±3.46ms 14.2 MB/sec fixed_size_binary_ree/parquet_2 1.00 82.3±0.69ms 12.2 MB/sec 1.02 84.2±0.48ms 12.0 MB/sec fixed_size_binary_ree/zstd 1.00 74.4±0.35ms 13.5 MB/sec 1.02 75.8±1.01ms 13.3 MB/sec fixed_size_binary_ree/zstd_parquet_2 1.00 83.9±0.89ms 12.0 MB/sec 1.02 85.3±1.43ms 11.8 MB/sec float_with_nans/bloom_filter 1.00 89.3±0.75ms 156.9 MB/sec 1.00 89.5±1.00ms 156.5 MB/sec float_with_nans/cdc 1.00 70.0±0.47ms 200.1 MB/sec 1.00 70.0±0.52ms 199.9 MB/sec float_with_nans/default 1.00 71.2±0.47ms 196.5 MB/sec 1.00 70.9±1.24ms 197.5 MB/sec float_with_nans/number_distinct_values 1.00 170.7±1.07ms 82.0 MB/sec 1.04 176.8±11.02ms 79.2 MB/sec float_with_nans/parquet_2 1.00 87.3±0.66ms 160.3 MB/sec 1.00 87.5±0.65ms 160.1 MB/sec float_with_nans/zstd 1.00 109.6±0.63ms 127.8 MB/sec 1.01 110.7±1.29ms 126.5 MB/sec float_with_nans/zstd_parquet_2 1.00 124.3±1.01ms 112.6 MB/sec 1.01 125.1±1.22ms 112.0 MB/sec fsb/bloom_filter 1.00 8.1±0.16ms 1995.2 MB/sec 1.05 8.5±0.27ms 1894.7 MB/sec fsb/cdc 1.00 10.6±0.11ms 1527.5 MB/sec 1.02 10.8±0.11ms 1495.4 MB/sec fsb/default 1.00 7.0±0.07ms 2.3 GB/sec 1.04 7.3±0.06ms 2.2 GB/sec fsb/number_distinct_values 1.00 11.3±0.19ms 1423.4 MB/sec 1.05 11.9±0.34ms 1352.4 MB/sec fsb/parquet_2 1.00 13.1±0.15ms 1234.1 MB/sec 1.05 13.7±0.15ms 1179.8 MB/sec fsb/zstd 1.00 9.4±0.12ms 1708.4 MB/sec 1.02 9.6±0.22ms 1674.8 MB/sec fsb/zstd_parquet_2 1.00 14.0±0.09ms 1148.5 MB/sec 1.03 14.4±0.27ms 1118.5 MB/sec int32_ree/bloom_filter 1.01 40.7±0.42ms 10.0 MB/sec 1.00 40.3±0.26ms 10.1 MB/sec int32_ree/cdc 1.00 39.7±0.11ms 10.2 MB/sec 1.00 39.7±0.10ms 10.2 MB/sec int32_ree/default 1.00 35.5±0.10ms 11.5 MB/sec 1.00 35.5±0.10ms 11.4 MB/sec int32_ree/number_distinct_values 1.00 52.5±0.09ms 7.7 MB/sec 1.02 53.6±3.25ms 7.6 MB/sec int32_ree/parquet_2 1.01 35.9±0.12ms 11.3 MB/sec 1.00 35.5±0.11ms 11.5 MB/sec int32_ree/zstd 1.01 36.6±0.11ms 11.1 MB/sec 1.00 36.1±0.08ms 11.3 MB/sec int32_ree/zstd_parquet_2 1.02 36.6±0.12ms 11.1 MB/sec 1.00 36.0±0.14ms 11.3 MB/sec int32_ree_95pct_null/bloom_filter 1.00 22.2±0.05ms 18.4 MB/sec 1.00 22.2±0.04ms 18.4 MB/sec int32_ree_95pct_null/cdc 1.00 23.9±0.06ms 17.0 MB/sec 1.02 24.5±0.04ms 16.6 MB/sec int32_ree_95pct_null/default 1.00 21.6±0.06ms 18.8 MB/sec 1.00 21.6±0.04ms 18.8 MB/sec int32_ree_95pct_null/number_distinct_values 1.00 22.6±0.05ms 18.0 MB/sec 1.00 22.7±0.16ms 17.9 MB/sec int32_ree_95pct_null/parquet_2 1.00 21.6±0.04ms 18.8 MB/sec 1.00 21.6±0.03ms 18.8 MB/sec int32_ree_95pct_null/zstd 1.00 21.9±0.06ms 18.5 MB/sec 1.00 21.9±0.04ms 18.6 MB/sec int32_ree_95pct_null/zstd_parquet_2 1.00 21.8±0.04ms 18.6 MB/sec 1.00 21.8±0.03ms 18.6 MB/sec large_string_distinct/delta_byte_array 1.00 63.3±1.73ms 4.0 GB/sec 1.04 65.9±2.87ms 3.8 GB/sec large_string_distinct/plain 1.00 61.7±1.32ms 4.1 GB/sec 1.02 62.9±1.42ms 4.0 GB/sec large_string_distinct_nullable/delta_byte_array 1.02 56.7±2.12ms 4.4 GB/sec 1.00 55.5±1.58ms 4.5 GB/sec large_string_non_null/bloom_filter 1.03 69.1±0.51ms 3.6 GB/sec 1.00 67.4±0.52ms 3.7 GB/sec large_string_non_null/cdc 1.01 240.4±1.04ms 1065.0 MB/sec 1.00 237.7±1.00ms 1076.8 MB/sec large_string_non_null/default 1.02 52.2±0.55ms 4.8 GB/sec 1.00 51.0±0.64ms 4.9 GB/sec large_string_non_null/number_distinct_values 1.03 69.2±0.38ms 3.6 GB/sec 1.00 67.0±0.40ms 3.7 GB/sec large_string_non_null/parquet_2 1.01 51.6±0.42ms 4.8 GB/sec 1.00 50.9±0.54ms 4.9 GB/sec large_string_non_null/zstd 1.04 52.2±0.57ms 4.8 GB/sec 1.00 50.0±0.22ms 5.0 GB/sec large_string_non_null/zstd_parquet_2 1.03 52.3±0.52ms 4.8 GB/sec 1.00 51.0±0.63ms 4.9 GB/sec large_string_shared_prefix/delta_byte_array 1.00 43.5±0.93ms 5.7 GB/sec 1.03 45.0±0.89ms 5.6 GB/sec large_string_shared_prefix/plain 1.00 94.4±1.35ms 2.6 GB/sec 1.06 100.1±2.96ms 2.5 GB/sec large_string_shared_prefix_list/delta_byte_array 1.00 56.9±5.45ms 4.4 GB/sec 1.02 58.3±1.38ms 4.3 GB/sec large_string_shared_prefix_nullable/delta_byte_array 1.00 41.0±0.73ms 6.1 GB/sec 1.09 44.6±0.76ms 5.6 GB/sec large_string_shared_prefix_nullable/plain 1.00 63.1±1.13ms 4.0 GB/sec 1.05 66.3±2.31ms 3.8 GB/sec large_string_shared_prefix_nullable_dense/delta_byte_array 1.00 21.9±0.24ms 5.7 GB/sec 1.05 23.0±0.46ms 5.4 GB/sec large_string_shared_prefix_nullable_trailing/delta_byte_array 1.00 40.5±0.52ms 6.2 GB/sec 1.80 72.8±2.80ms 3.4 GB/sec list_nested/bloom_filter 1.03 148.6±0.35ms 194.2 MB/sec 1.00 144.2±0.81ms 200.1 MB/sec list_nested/cdc 1.04 166.6±0.47ms 173.2 MB/sec 1.00 160.3±0.36ms 180.0 MB/sec list_nested/default 1.05 135.6±0.34ms 212.8 MB/sec 1.00 129.3±0.43ms 223.2 MB/sec list_nested/number_distinct_values 1.00 308.0±5.62ms 93.7 MB/sec 1.02 314.3±32.08ms 91.8 MB/sec list_nested/parquet_2 1.05 148.9±1.11ms 193.8 MB/sec 1.00 142.4±0.42ms 202.6 MB/sec list_nested/zstd 1.05 144.8±0.40ms 199.3 MB/sec 1.00 138.3±0.53ms 208.6 MB/sec list_nested/zstd_parquet_2 1.04 161.5±0.37ms 178.6 MB/sec 1.00 155.3±0.42ms 185.7 MB/sec list_primitive/bloom_filter 1.09 306.5±24.64ms 1779.3 MB/sec 1.00 282.2±5.11ms 1932.4 MB/sec list_primitive/cdc 1.01 316.4±8.01ms 1723.8 MB/sec 1.00 313.3±2.59ms 1740.8 MB/sec list_primitive/default 1.05 222.2±20.50ms 2.4 GB/sec 1.00 211.1±18.67ms 2.5 GB/sec list_primitive/number_distinct_values 1.00 402.5±9.88ms 1354.8 MB/sec 1.04 420.4±18.51ms 1297.4 MB/sec list_primitive/parquet_2 1.33 294.0±6.02ms 1854.8 MB/sec 1.00 221.7±1.14ms 2.4 GB/sec list_primitive/zstd 1.07 488.0±4.63ms 1117.5 MB/sec 1.00 456.5±3.68ms 1194.7 MB/sec list_primitive/zstd_parquet_2 1.09 485.7±1.95ms 1122.8 MB/sec 1.00 444.3±1.96ms 1227.5 MB/sec list_primitive_non_null/bloom_filter 1.00 380.8±14.57ms 1429.1 MB/sec 1.01 384.0±17.65ms 1417.2 MB/sec list_primitive_non_null/cdc 1.00 386.9±3.80ms 1406.8 MB/sec 1.01 389.1±7.50ms 1398.8 MB/sec list_primitive_non_null/default 1.00 240.1±8.30ms 2.2 GB/sec 1.00 241.3±12.05ms 2.2 GB/sec list_primitive_non_null/number_distinct_values 1.00 592.1±19.79ms 919.1 MB/sec 1.01 600.5±33.42ms 906.2 MB/sec list_primitive_non_null/parquet_2 1.00 253.5±12.43ms 2.1 GB/sec 1.02 258.8±11.99ms 2.1 GB/sec list_primitive_non_null/zstd 1.00 668.6±5.85ms 814.0 MB/sec 1.00 666.5±6.25ms 816.6 MB/sec list_primitive_non_null/zstd_parquet_2 1.00 636.9±14.71ms 854.6 MB/sec 1.01 645.3±13.60ms 843.3 MB/sec list_primitive_sparse_99pct_null/bloom_filter 1.00 13.0±0.13ms 2.8 GB/sec 1.00 13.0±0.12ms 2.8 GB/sec list_primitive_sparse_99pct_null/cdc 1.00 24.4±0.12ms 1530.4 MB/sec 1.02 24.8±0.36ms 1504.6 MB/sec list_primitive_sparse_99pct_null/default 1.01 12.7±0.11ms 2.9 GB/sec 1.00 12.6±0.15ms 2.9 GB/sec list_primitive_sparse_99pct_null/number_distinct_values 1.00 12.6±0.10ms 2.9 GB/sec 1.02 12.8±0.11ms 2.9 GB/sec list_primitive_sparse_99pct_null/parquet_2 1.00 12.6±0.08ms 2.9 GB/sec 1.01 12.8±0.34ms 2.9 GB/sec list_primitive_sparse_99pct_null/zstd 1.00 14.5±0.09ms 2.5 GB/sec 1.01 14.7±0.27ms 2.5 GB/sec list_primitive_sparse_99pct_null/zstd_parquet_2 1.00 12.5±0.14ms 2.9 GB/sec 1.04 13.0±0.07ms 2.8 GB/sec list_struct_with_list/bloom_filter 1.09 305.3±5.31ms 149.3 MB/sec 1.00 280.1±1.93ms 162.8 MB/sec list_struct_with_list/cdc 1.07 340.4±8.98ms 133.9 MB/sec 1.00 318.7±1.52ms 143.1 MB/sec list_struct_with_list/default 1.11 280.5±7.07ms 162.6 MB/sec 1.00 253.3±1.13ms 180.0 MB/sec list_struct_with_list/number_distinct_values 1.01 572.6±8.82ms 79.6 MB/sec 1.00 569.0±33.96ms 80.1 MB/sec list_struct_with_list/parquet_2 1.09 295.2±6.98ms 154.5 MB/sec 1.00 270.9±1.56ms 168.3 MB/sec list_struct_with_list/zstd 1.09 300.2±8.36ms 151.9 MB/sec 1.00 275.9±1.57ms 165.3 MB/sec list_struct_with_list/zstd_parquet_2 1.09 318.4±10.24ms 143.2 MB/sec 1.00 293.4±1.76ms 155.4 MB/sec ``` </p> </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 2125.5s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 2032.0s | | CPU sys | 89.3s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 2155.5s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 2012.1s | | CPU sys | 138.4s | | Peak spill | 0 B | </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
