alamb commented on PR #7111: URL: https://github.com/apache/arrow-rs/pull/7111#issuecomment-2762417125
I ran the performance benchmarks and I do not see any changes for parquet writing with this PR (though the numbers do vary somewhat). Nice work. Details below <details><summary>Details</summary> <p> ``` ++ critcmp main encryption-basics-fork group encryption-basics-fork main ----- ---------------------- ---- write_batch nested/4096 values primitive list 1.00 6.5±0.30ms 325.6 MB/sec 1.03 6.7±0.26ms 316.1 MB/sec write_batch nested/4096 values primitive list non-null 1.00 8.2±0.37ms 260.8 MB/sec 1.00 8.2±0.29ms 260.1 MB/sec write_batch primitive/4096 values bool 1.01 1022.2±53.98µs 1062.3 KB/sec 1.00 1013.0±40.47µs 1072.0 KB/sec write_batch primitive/4096 values bool non-null 1.00 934.1±27.14µs 627.3 KB/sec 1.03 960.6±34.00µs 610.0 KB/sec write_batch primitive/4096 values float with NaNs 1.01 1824.6±56.19µs 30.1 MB/sec 1.00 1809.3±31.57µs 30.4 MB/sec write_batch primitive/4096 values primitive 1.04 2.9±0.24ms 60.4 MB/sec 1.00 2.8±0.07ms 62.6 MB/sec write_batch primitive/4096 values primitive non-null 1.06 2.9±0.25ms 60.5 MB/sec 1.00 2.7±0.13ms 64.0 MB/sec write_batch primitive/4096 values primitive non-null with bloom filter 1.05 22.5±2.68ms 7.7 MB/sec 1.00 21.5±0.36ms 8.0 MB/sec write_batch primitive/4096 values primitive with bloom filter 1.05 22.4±3.68ms 7.8 MB/sec 1.00 21.4±0.88ms 8.2 MB/sec write_batch primitive/4096 values string 1.00 1746.6±55.52µs 72.3 MB/sec 1.01 1757.4±46.59µs 71.8 MB/sec write_batch primitive/4096 values string dictionary 1.02 4.3±0.40ms 241.6 MB/sec 1.00 4.2±0.18ms 247.1 MB/sec write_batch primitive/4096 values string dictionary with bloom filter 1.02 5.8±0.33ms 177.5 MB/sec 1.00 5.7±0.27ms 180.3 MB/sec write_batch primitive/4096 values string non-null 1.02 6.9±0.24ms 295.6 MB/sec 1.00 6.8±0.25ms 302.8 MB/sec write_batch primitive/4096 values string non-null with bloom filter 1.00 12.9±0.34ms 159.3 MB/sec 1.00 12.9±0.36ms 158.6 MB/sec write_batch primitive/4096 values string with bloom filter 1.00 11.1±0.63ms 184.1 MB/sec 1.01 11.2±0.37ms 182.3 MB/sec ``` Second run ``` ++ critcmp main encryption-basics-fork group encryption-basics-fork main ----- ---------------------- ---- write_batch nested/4096 values primitive list 1.01 6.9±0.29ms 309.3 MB/sec 1.00 6.8±0.27ms 313.5 MB/sec write_batch nested/4096 values primitive list non-null 1.00 8.5±0.41ms 251.6 MB/sec 1.04 8.8±0.51ms 241.4 MB/sec write_batch primitive/4096 values bool 1.00 1012.3±34.35µs 1072.7 KB/sec 1.03 1044.5±39.21µs 1039.6 KB/sec write_batch primitive/4096 values bool non-null 1.00 945.6±33.77µs 619.6 KB/sec 1.02 962.4±43.08µs 608.8 KB/sec write_batch primitive/4096 values float with NaNs 1.01 1876.1±54.31µs 29.3 MB/sec 1.00 1861.8±63.39µs 29.5 MB/sec write_batch primitive/4096 values primitive 1.03 3.0±0.13ms 58.5 MB/sec 1.00 2.9±0.12ms 60.4 MB/sec write_batch primitive/4096 values primitive non-null 1.04 3.0±0.75ms 57.7 MB/sec 1.00 2.9±0.60ms 60.0 MB/sec write_batch primitive/4096 values primitive non-null with bloom filter 1.00 21.5±0.50ms 8.0 MB/sec 1.00 21.5±0.65ms 8.0 MB/sec write_batch primitive/4096 values primitive with bloom filter 1.00 21.4±0.85ms 8.2 MB/sec 1.00 21.4±0.96ms 8.2 MB/sec write_batch primitive/4096 values string 1.00 1790.4±68.70µs 70.5 MB/sec 1.02 1827.0±87.24µs 69.1 MB/sec write_batch primitive/4096 values string dictionary 1.00 4.6±0.28ms 225.3 MB/sec 1.06 4.8±0.30ms 213.4 MB/sec write_batch primitive/4096 values string dictionary with bloom filter 1.00 6.2±0.31ms 166.9 MB/sec 1.01 6.2±0.42ms 165.3 MB/sec write_batch primitive/4096 values string non-null 1.00 7.2±0.29ms 285.1 MB/sec 1.09 7.8±0.54ms 261.6 MB/sec write_batch primitive/4096 values string non-null with bloom filter 1.01 13.4±0.41ms 152.7 MB/sec 1.00 13.3±0.42ms 153.9 MB/sec write_batch primitive/4096 values string with bloom filter 1.00 11.9±0.48ms 172.3 MB/sec 1.00 11.9±0.35ms 172.7 MB/sec ``` </p> </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
