sdf-jkl commented on code in PR #11411:
URL: https://github.com/apache/arrow-rs/pull/11411#discussion_r4211311123
##########
parquet-variant-compute/benches/variant_kernels.rs:
##########
Review Comment:
and it covers some of the stuff that this PR is introducing:
- short unescaped strings
- nested cases
- decimals have some common cases, but some are a valid introduction for the
decimal parsing rework
- int's only useful addition is the Int32 case
##########
parquet-variant-compute/benches/variant_kernels.rs:
##########
@@ -530,6 +582,7 @@ criterion_group!(
shred_variant_partial_object_bench,
shred_variant_unmatched_object_bench,
variant_array_builder_build_bench,
+ benchmark_json_to_variant_inputs,
benchmark_batch_json_string_to_variant,
Review Comment:
It seems like there's an already preexisting `json_to_variant` kernel
benchmark
##########
parquet-variant-compute/benches/variant_kernels.rs:
##########
@@ -42,6 +42,58 @@ const VARIANT_GET_UNSHREDDED_OBJECT_ROWS: usize = 262_144;
const VARIANT_ARRAY_BUILD_ROWS: usize = 262_144;
const SHRED_VARIANT_OBJECT_ROWS: usize = 8_192;
+fn benchmark_json_to_variant_inputs(c: &mut Criterion) {
+ let large_array = format!(
+ "[{}]",
+ (0..1024)
+ .map(|number| number.to_string())
+ .collect::<Vec<_>>()
+ .join(",")
+ );
+ let large_string = format!(r#"{{"payload":"{}","id":42}}"#,
"x".repeat(8192));
+ let duplicate_large = format!(r#"{{"a":"{}","a":0}}"#,
"x".repeat(100_000));
+ let inputs = [
+ (
+ "integers",
+ r#"{"id":123456789,"values":[1,2,3,4,5],"active":true}"#,
+ ),
+ (
+ "decimals",
+
r#"{"small":1.23,"medium":999999999.0,"large":0.9999999999999999999}"#,
+ ),
+ (
+ "strings",
+ r#"{"first":"alpha","second":"beta","third":"gamma"}"#,
+ ),
+ (
+ "escaped_strings",
+ r#"{"line":"one\ntwo","quote":"\"value\"","unicode":"\u2764"}"#,
+ ),
+ (
+ "nested",
+
r#"{"outer":[{"id":1,"values":[1.25,2.50]},{"id":2,"values":[3.75,4.00]}]}"#,
+ ),
+ ("large_array_1024", large_array.as_str()),
+ ("large_string_8k", large_string.as_str()),
+ ("duplicate_key_100k", duplicate_large.as_str()),
+ ];
+
+ let mut group = c.benchmark_group("json_to_variant_inputs");
Review Comment:
groups and throughput do not follow the same format the rests of the benches
use
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
