Baunsgaard commented on PR #15640: URL: https://github.com/apache/iceberg/pull/15640#issuecomment-4261376094
Just to help out a bit I ran the benchmark. While there are no practical difference end-to-end since other things dominate, it is a minor but valid improvement. ## ManifestReadBenchmark Results OpenJDK 17.0.18, Ubuntu 22.04, warmup=5, measure=10 iterations. ### End-to-end: read manifest list + iterate all data files | Size | LinkedList | ArrayList | Ratio | |-----------------------------|-------------|-------------|-------| | 5 manifests × 100 rows | 9.70 ms | 9.48 ms | 1.02x | | 50 manifests × 1K rows | 316.5 ms | 317.8 ms | 1.00x | | 200 manifests × 10K rows | 11,827 ms | 11,820 ms | 1.00x | ### Materialization only (ManifestLists.read) | Size | LinkedList | ArrayList | Ratio | |---------------|------------|-----------|-------| | 5 manifests | 868 us | 841 us | 1.03x | | 50 manifests | 864 us | 835 us | 1.04x | | 200 manifests | 1.14 ms | 1.15 ms | 0.99x | ## Extra specific measurements: ### Iteration after materialization | Size | LinkedList | ArrayList | Ratio | |---------------|------------|-----------|-----------| | 5 manifests | 1.9 us | 730 ns | **2.60x** | | 50 manifests | 12.2 us | 6.9 us | **1.76x** | | 200 manifests | 25.9 us | 23.2 us | 1.12x | ### addAll into ArrayList (FastAppend.apply pattern) | Size | LinkedList | ArrayList | Ratio | |---------------|------------|-----------|-----------| | 5 manifests | 847 ns | 546 ns | **1.55x** | | 50 manifests | 1.3 us | 486 ns | **2.59x** | | 200 manifests | 3.8 us | 523 ns | **7.33x** | ### Full pipeline: read + filter + ImmutableList.copyOf | Size | LinkedList | ArrayList | Ratio | |---------------|------------|-----------|-------| | 5 manifests | 409 us | 422 us | 0.97x | | 50 manifests | 387 us | 390 us | 0.99x | | 200 manifests | 566 us | 581 us | 0.97x | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
