marklit commented on issue #15220: URL: https://github.com/apache/arrow/issues/15220#issuecomment-1435922148
I don't have any way of adjusting ClickHouse's compression settings. If I produce a 1-row PQ file with PyArrow and again with ClickHouse I can see the headers are different (PyArrow's are much longer) and the above PQ files produced were off by a few MBs. I'm not sure if it is possible to produce byte-identical PQ files with both tools. With that said, I'm not convinced it's down to Snappy being well-optimised in ClickHouse and unoptimised in PyArrow. Snappy shows up 4.8% of the time in the Flamegraph. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
