Yibo Cai created ARROW-11032: -------------------------------- Summary: [C++][Flight] Add benchmark for local RPC through unix socket Key: ARROW-11032 URL: https://issues.apache.org/jira/browse/ARROW-11032 Project: Apache Arrow Issue Type: Improvement Components: FlightRPC Reporter: Yibo Cai Assignee: Yibo Cai
For local RPC, unix socket is more efficient than TCP socket through localhost. Benchmark shows it has higher throughput and lower latency. According patch adds "-test_unix" option to enable unix socket based test. *One thread (speed 2992 -> 3419, latency 37 -> 35)* {code:bash} # one thread, TCP socket $ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100 Using spawned TCP server Server running with pid 13741 Server host: localhost Server port: 31337 Testing method: DoGet Server host: localhost Server port: 31337 Number of perf runs: 100 Number of concurrent gets/puts: 1 Batch size: 131040 Batches read: 976800 Bytes read: 128000000000 Nanos: 40786538125 Speed: 2992.91 MB/s Throughput: 23949.1 batches/s Latency mean: 38 us Latency quantile=0.5: 37 us Latency quantile=0.95: 86 us Latency quantile=0.99: 446 us Latency max: 4747 us # one thread, Unix socket $ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100 --test-unix Using spawned Unix server Server running with pid 13610 Server unix socket: /tmp/flight-bench-spawn.sock Testing method: DoGet Server unix socket: /tmp/flight-bench-spawn.sock Number of perf runs: 100 Number of concurrent gets/puts: 1 Batch size: 131040 Batches read: 976800 Bytes read: 128000000000 Nanos: 35700637793 Speed: 3419.28 MB/s Throughput: 27360.9 batches/s Latency mean: 35 us Latency quantile=0.5: 25 us Latency quantile=0.95: 104 us Latency quantile=0.99: 401 us Latency max: 6943 us {code} *Four threads (speed 9096 -> 10067, latency 49 -> 44)* {code:bash} # four threads, TCP socket $ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100 Using spawned TCP server Server running with pid 13860 Server host: localhost Server port: 31337 Testing method: DoGet Server host: localhost Server port: 31337 Number of perf runs: 100 Number of concurrent gets/puts: 4 Batch size: 131040 Batches read: 976800 Bytes read: 128000000000 Nanos: 13419015574 Speed: 9096.82 MB/s Throughput: 72792.2 batches/s Latency mean: 49 us Latency quantile=0.5: 38 us Latency quantile=0.95: 69 us Latency quantile=0.99: 253 us Latency max: 8954 us # four threads, Unix socket $ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100 --test-unix Using spawned Unix server Server running with pid 14587 Server unix socket: /tmp/flight-bench-spawn.sock Testing method: DoGet Server unix socket: /tmp/flight-bench-spawn.sock Number of perf runs: 100 Number of concurrent gets/puts: 4 Batch size: 131040 Batches read: 976800 Bytes read: 128000000000 Nanos: 12125043167 Speed: 10067.6 MB/s Throughput: 80560.5 batches/s Latency mean: 44 us Latency quantile=0.5: 5 us Latency quantile=0.95: 47 us Latency quantile=0.99: 1378 us Latency max: 8359 us {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)