osipovartem opened a new pull request, #11411:
URL: https://github.com/apache/arrow-rs/pull/11411
## Which issue does this PR relate to?
- Benchmark split requested in #11177 (related to #11169).
- Provides a baseline for the direct JSON parser proposed in #11409. This PR
is independent of that parser and the decimal-conversion patch.
## What changes are included?
Add a Criterion benchmark for three existing paths in \:
- \: JSON text to \;
- \: an already-parsed Value to Variant;
- \: the public JSON string to Variant API end to end.
Inputs cover integers, decimals, strings and escapes, nested objects, a
1,024-element array, an 8 KiB string, and a 100 KiB duplicate-key case. Byte
throughput is reported only for stages that read the source JSON; the Value
conversion stage reports time only because duplicate-key parsing can collapse
the input before conversion.
No library behavior or public API changes.
## Validation
- \
- \json_parse/integers time: [223.63 ns 226.78 ns 227.57 ns]
thrpt: [213.73 MiB/s 214.47 MiB/s 217.49 MiB/s]
json_parse/decimals time: [181.65 ns 181.71 ns 181.96 ns]
thrpt: [340.67 MiB/s 341.14 MiB/s 341.26 MiB/s]
json_parse/strings time: [157.97 ns 160.32 ns 160.91 ns]
thrpt: [290.41 MiB/s 291.47 MiB/s 295.81 MiB/s]
json_parse/escaped_strings
time: [186.17 ns 187.27 ns 187.54 ns]
thrpt: [294.94 MiB/s 295.37 MiB/s 297.11 MiB/s]
json_parse/nested time: [342.80 ns 345.99 ns 346.79 ns]
thrpt: [195.25 MiB/s 195.70 MiB/s 197.52 MiB/s]
json_parse/large_array_1024
time: [11.313 µs 11.391 µs 11.701 µs]
thrpt: [326.91 MiB/s 335.82 MiB/s 338.13 MiB/s]
json_parse/large_string_8k
time: [771.19 ns 772.99 ns 773.44 ns]
thrpt: [9.8908 GiB/s 9.8965 GiB/s 9.9196 GiB/s]
json_parse/duplicate_key_100k
time: [8.5739 µs 8.5822 µs 8.6155 µs]
thrpt: [10.811 GiB/s 10.853 GiB/s 10.864 GiB/s]
variant_from_json_value/integers
time: [441.64 ns 443.58 ns 451.36 ns]
variant_from_json_value/decimals
time: [343.07 ns 343.69 ns 346.18 ns]
variant_from_json_value/strings
time: [365.47 ns 370.15 ns 371.32 ns]
variant_from_json_value/escaped_strings
time: [355.34 ns 358.71 ns 372.17 ns]
variant_from_json_value/nested
time: [865.28 ns 878.83 ns 882.22 ns]
variant_from_json_value/large_array_1024
time: [14.677 µs 14.867 µs 14.915 µs]
variant_from_json_value/large_string_8k
time: [411.63 ns 412.77 ns 417.30 ns]
variant_from_json_value/duplicate_key_100k
time: [154.33 ns 154.58 ns 155.58 ns]
variant_from_json_string/integers
time: [863.97 ns 875.26 ns 878.08 ns]
thrpt: [55.390 MiB/s 55.569 MiB/s 56.295 MiB/s]
variant_from_json_string/decimals
time: [534.70 ns 544.50 ns 546.95 ns]
thrpt: [113.34 MiB/s 113.85 MiB/s 115.93 MiB/s]
variant_from_json_string/strings
time: [521.80 ns 543.77 ns 549.26 ns]
thrpt: [85.078 MiB/s 85.937 MiB/s 89.556 MiB/s]
variant_from_json_string/escaped_strings
time: [546.34 ns 567.29 ns 572.53 ns]
thrpt: [96.611 MiB/s 97.503 MiB/s 101.24 MiB/s]
variant_from_json_string/nested
time: [1.2264 µs 1.2399 µs 1.2432 µs]
thrpt: [54.463 MiB/s 54.611 MiB/s 55.210 MiB/s]
variant_from_json_string/large_array_1024
time: [26.954 µs 27.049 µs 27.072 µs]
thrpt: [141.30 MiB/s 141.42 MiB/s 141.92 MiB/s]
variant_from_json_string/large_string_8k
time: [1.1846 µs 1.1861 µs 1.1865 µs]
thrpt: [6.4475 GiB/s 6.4495 GiB/s 6.4575 GiB/s]
variant_from_json_string/duplicate_key_100k
time: [8.8321 µs 8.8925 µs 8.9076 µs]
thrpt: [10.457 GiB/s 10.475 GiB/s 10.546 GiB/s]
(all 24 cases ran)
- \
running 66 tests
test from_json::test::test_json_to_variant_array_simple ... ok
test from_json::test::test_json_to_variant_decimal16_high_precision ...
ignored
test from_json::test::test_json_to_variant_decimal16_large_integer ...
ignored
test from_json::test::test_json_to_variant_decimal16_max_scale ... ignored
test from_json::test::test_json_to_variant_decimal16_max_value ... ignored
test from_json::test::test_json_to_variant_decimal4_basic ... ignored
test from_json::test::test_json_to_variant_decimal4_large_negative ...
ignored
test from_json::test::test_json_to_variant_decimal4_large_positive ...
ignored
test from_json::test::test_json_to_variant_decimal4_small_negative ...
ignored
test from_json::test::test_json_to_variant_decimal4_small_positive ...
ignored
test from_json::test::test_json_to_variant_decimal4_tiny_positive ... ignored
test from_json::test::test_json_to_variant_decimal8_high_precision ...
ignored
test
from_json::test::test_json_to_variant_decimal8_large_negative_with_scale ...
ignored
test from_json::test::test_json_to_variant_decimal8_large_with_scale ...
ignored
test from_json::test::test_json_to_variant_decimal8_negative ... ignored
test from_json::test::test_json_to_variant_decimal8_positive ... ignored
test from_json::test::test_json_to_variant_boolean_false ... ok
test from_json::test::test_json_to_variant_array_large_u16_offset ... ok
test from_json::test::test_json_to_variant_array_with_object ... ok
test from_json::test::test_json_to_variant_double_precision ... ok
test from_json::test::test_json_to_variant_boolean_true ... ok
test from_json::test::test_json_to_variant_double_scientific_negative ... ok
test from_json::test::test_json_to_variant_double_scientific_positive ... ok
test from_json::test::test_json_to_variant_int16 ... ok
test from_json::test::test_json_to_variant_int32 ... ok
test from_json::test::test_json_to_variant_int8_negative ... ok
test from_json::test::test_json_to_variant_int64 ... ok
test from_json::test::test_json_to_variant_int8_positive ... ok
test from_json::test::test_json_to_variant_long_string ... ok
test from_json::test::test_json_to_variant_null ... ok
test from_json::test::test_json_to_variant_object_simple ... ok
test from_json::test::test_json_to_variant_object_complex ... ok
test from_json::test::test_json_to_variant_short_string ... ok
test from_json::test::test_json_to_variant_short_string_max_length ... ok
test from_json::test::test_json_to_variant_unicode ... ok
test to_json::tests::test_binary_to_json ... ok
test to_json::tests::test_buffer_writing_variants ... ok
test to_json::tests::test_date_to_json ... ok
test from_json::test::test_json_to_variant_very_long_string ... ok
test to_json::tests::test_decimal16_to_json ... ok
test to_json::tests::test_decimal_edge_cases ... ok
test to_json::tests::test_decimal_precision_behavior ... ok
test to_json::tests::test_empty_list_to_json ... ok
test to_json::tests::test_empty_object_to_json ... ok
test to_json::tests::test_float_nan_inf_handling ... ok
test to_json::tests::test_json_buffer_writing ... ok
test to_json::tests::test_list_with_various_primitive_types_to_json ... ok
test to_json::tests::test_mixed_type_list_to_json ... ok
test to_json::tests::test_object_field_ordering_in_json ... ok
test to_json::tests::test_object_with_special_characters_to_json ... ok
test to_json::tests::test_object_with_various_primitive_types_to_json ... ok
test to_json::tests::test_primitive_json_conversion ... ok
test to_json::tests::test_short_string_to_json ... ok
test to_json::tests::test_simple_list_to_json ... ok
test to_json::tests::test_simple_object_to_json ... ok
test to_json::tests::test_string_escaping ... ok
test to_json::tests::test_string_escaping_comprehensive ... ok
test to_json::tests::test_string_to_json ... ok
test to_json::tests::test_time_to_json ... ok
test to_json::tests::test_timestamp_micros_to_json ... ok
test to_json::tests::test_timestamp_nanos_to_json ... ok
test to_json::tests::test_timestamp_ntz_micros_to_json ... ok
test to_json::tests::test_timestamp_ntz_nanos_to_json ... ok
test to_json::tests::test_uuid_to_json ... ok
test from_json::test::test_json_to_variant_array_nested_large ... ok
test from_json::test::test_json_to_variant_object_very_large ... ok
test result: ok. 51 passed; 0 failed; 15 ignored; 0 measured; 0 filtered
out; finished in 17.08s
running 6 tests
test parquet-variant-json/src/to_json.rs - to_json::VariantToJson::to_json
(line 60) ... ok
test parquet-variant-json/src/to_json.rs - to_json::VariantToJson::to_json
(line 48) ... ok
test parquet-variant-json/src/from_json.rs - from_json::JsonToVariant (line
40) ... ok
test parquet-variant-json/src/to_json.rs -
to_json::VariantToJson::to_json_string (line 98) ... ok
test parquet-variant-json/src/to_json.rs -
to_json::VariantToJson::to_json_string (line 118) ... ok
test parquet-variant-json/src/to_json.rs -
to_json::VariantToJson::to_json_value (line 155) ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out;
finished in 0.00s
all doctests ran in 0.40s; merged doctests compilation took 0.40s (51 tests
and 6 doctests passed; 15 existing decimal tests ignored)
- \
- Independent read-only review: approved, no blocking findings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
