andygrove commented on code in PR #4918:
URL: https://github.com/apache/datafusion-comet/pull/4918#discussion_r3603867263
##########
native/spark-expr/src/conversion_funcs/numeric.rs:
##########
@@ -155,53 +157,65 @@ macro_rules! cast_float_to_string {
const LOWER_SCIENTIFIC_BOUND: $type = 0.001;
const UPPER_SCIENTIFIC_BOUND: $type = 10000000.0;
- let output_array = array
- .iter()
- .map(|value| match value {
- Some(value) if value == <$type>::INFINITY =>
Ok(Some("Infinity".to_string())),
- Some(value) if value == <$type>::NEG_INFINITY =>
Ok(Some("-Infinity".to_string())),
- Some(value)
- if (value.abs() < UPPER_SCIENTIFIC_BOUND
- && value.abs() >= LOWER_SCIENTIFIC_BOUND)
- || value.abs() == 0.0 =>
- {
- let trailing_zero = if value.fract() == 0.0 { ".0"
} else { "" };
-
- Ok(Some(format!("{value}{trailing_zero}")))
+ // Values are formatted straight into the builder, so no
intermediate String
+ // is allocated per row.
+ let mut builder =
GenericStringBuilder::<OffsetSize>::with_capacity(
+ array.len(),
+ array.len() * 8,
Review Comment:
Raised the value-buffer hint to `array.len() * 16` to match arrow-rs's own
`AVERAGE_STRING_LENGTH`, so typical outputs like `1234.5678` and `-1.4E-45` no
longer force a mid-loop grow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
