Yibo Cai created ARROW-12581: -------------------------------- Summary: [C++][FlightRPC] Benchmark compression with real data Key: ARROW-12581 URL: https://issues.apache.org/jira/browse/ARROW-12581 Project: Apache Arrow Issue Type: Improvement Components: C++, FlightRPC Reporter: Yibo Cai
Flight benchmark program is using random test data as payload. Random data is hard to compress. Per my test, the compressed payload (zstd default level) is actually a bit larger than the original uncompressed payload. So compression is a pure loss in current benchmark. It's better to use real world test set for a more reasonable benchmark about how compression influences FlightRPC performance. Perhaps the benchmark (both client/server) could accept a path to an IPC file as an option. -- This message was sent by Atlassian Jira (v8.3.4#803005)