taepper commented on PR #51396: URL: https://github.com/apache/arrow/pull/51396#issuecomment-6518332926
I tested the 3 different implementations (RapidJSON, column-wise simdjson, row-wise simdjson): Median wall time (Ryzen 7 PRO 8700GE, GCC 13, Arrow release flags, single thread). | Dataset | RapidJSON (main) | simdjson, column-wise (a5d38cd523) | simdjson, row-wise (https://github.com/taepper/arrow/commit/5247184017656960f6615a4f3befca1ca0f6fe9b) | |---|---:|---:|---:| | **JSON -> Arrow** (parse + convert) | | | | | [twitter.json](https://github.com/simdjson/simdjson/blob/master/jsonexamples/twitter.json) (100 statuses as rows, 0.47 MB) | 1.61 ms (wrong result) | 0.68 ms | **0.61 ms** | | [canada.json](https://github.com/boostorg/json/blob/develop/bench/data/canada.json) (1 feature as row, 2.1 MB, 111k doubles in 3-level nested lists) | segfault | 9.50 ms | **3.01 ms** | | example records (the 3 example rows repeated to 100k rows, 10.8 MB) | 320 ms | 133 ms | **21.6 ms** | | **Arrow -> JSON** (to strings) | | | | | twitter.json | 1.52 ms | 0.69 ms | 0.84 ms | | canada.json | segfault | 41.7 ms | 41.6 ms | | example records | 21,325 ms | 33.9 ms | 35.4 ms | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
