Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-19 Thread via GitHub 


2010YOUY01 closed issue #22188: panic: capacity overflow in generate_series / 
range with large i64 arguments
URL: https://github.com/apache/datafusion/issues/22188


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-17 Thread via GitHub 


sweb commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4472339975

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-16 Thread via GitHub 


Dandandan commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4466608997

   Generated Code:
   ```
   // Licensed to the Apache Software Foundation (ASF) under one
   // or more contributor license agreements.  See the NOTICE file
   // distributed with this work for additional information
   // regarding copyright ownership.  The ASF licenses this file
   // to you under the Apache License, Version 2.0 (the
   // "License"); you may not use this file except in compliance
   // with the License.  You may obtain a copy of the License at
   //
   //   http://www.apache.org/licenses/LICENSE-2.0
   //
   // Unless required by applicable law or agreed to in writing,
   // software distributed under the License is distributed on an
   // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   // KIND, either express or implied.  See the License for the
   // specific language governing permissions and limitations
   // under the License.
   
   #![no_main]
   
   use std::sync::Arc;
   use std::sync::OnceLock;
   use std::time::Duration;
   
   use datafusion::arrow::array::{
   BooleanArray, Float64Array, Int32Array, Int64Array, StringArray,
   TimestampNanosecondArray,
   };
   use datafusion::arrow::datatypes::{DataType, Field, Schema, TimeUnit};
   use datafusion::arrow::record_batch::RecordBatch;
   use datafusion::catalog::MemTable;
   use datafusion::error::Result;
   use datafusion::execution::context::SessionContext;
   use datafusion::prelude::SessionConfig;
   use datafusion_common::instant::Instant;
   use libfuzzer_sys::fuzz_target;
   
   const DEFAULT_PLANNING_TIMEOUT_MS: u64 = 1_000;
   
   fn runtime() -> &'static tokio::runtime::Runtime {
   static RT: OnceLock = OnceLock::new();
   RT.get_or_init(|| {
   tokio::runtime::Builder::new_current_thread()
   .enable_time()
   .build()
   .expect("failed to create tokio runtime")
   })
   }
   
   fn planning_timeout() -> Duration {
   static T: OnceLock = OnceLock::new();
   *T.get_or_init(|| {
   std::env::var("DATAFUSION_FUZZ_PLANNING_TIMEOUT_MS")
   .ok()
   .and_then(|value| value.parse::().ok())
   .map(Duration::from_millis)
   .unwrap_or_else(|| 
Duration::from_millis(DEFAULT_PLANNING_TIMEOUT_MS))
   })
   }
   
   fuzz_target!(|data: &[u8]| {
   let sql = String::from_utf8_lossy(data);
   let started = Instant::now();
   
   runtime().block_on(async {
   // SQL, schema, and physical planning errors are expected fuzzer 
noise.
   // Panics and slow planning are the findings this target reports.
   let _ = plan_sql(&sql).await;
   });
   
   let elapsed = started.elapsed();
   let limit = planning_timeout();
   if elapsed > limit {
   eprintln!(
   "query exceeded {} ms after {} ms for SQL:\n{}",
   limit.as_millis(),
   elapsed.as_millis(),
   sql
   );
   panic!(
   "query exceeded {} ms after {} ms",
   limit.as_millis(),
   elapsed.as_millis()
   );
   }
   });
   
   async fn plan_sql(sql: &str) -> Result<()> {
   let ctx = fuzz_context()?;
   let dataframe = ctx.sql(sql).await?;
   let _batches = dataframe.collect().await?;
   Ok(())
   }
   
   fn fuzz_context() -> Result {
   let config = SessionConfig::new()
   .with_information_schema(true)
   .with_target_partitions(1);
   let ctx = SessionContext::new_with_config(config);
   
   register_demo_table(&ctx, "t")?;
   register_demo_table(&ctx, "t1")?;
   register_demo_table(&ctx, "t2")?;
   
   Ok(ctx)
   }
   
   fn register_demo_table(ctx: &SessionContext, table_name: &str) -> Result<()> 
{
   let schema = Arc::new(Schema::new(vec![
   Field::new("id", DataType::Int64, false),
   Field::new("a", DataType::Int64, true),
   Field::new("b", DataType::Float64, true),
   Field::new("c", DataType::Utf8, true),
   Field::new("x", DataType::Int32, true),
   Field::new("y", DataType::Boolean, true),
   Field::new("ts", DataType::Timestamp(TimeUnit::Nanosecond, None), 
true),
   ]));
   let id = Int64Array::from(vec![1, 2, 3, 4, 5]);
   let a = Int64Array::from(vec![Some(10), None, Some(-3), Some(0), 
Some(i64::MAX)]);
   let b = Float64Array::from(vec![
   Some(1.5),
   Some(-2.5),
   None,
   Some(0.0),
   Some(f64::INFINITY),
   ]);
   let c = StringArray::from(vec![Some("alpha"), Some(""), None, Some("z"), 
Some("héllo")]);
   let x = Int32Array::from(vec![Some(100), Some(-1), Some(0), None, 
Some(i32::MIN)]);
   let y = BooleanArray::from(vec![Some(true), Some(false), None, 
Some(true), Some(false)]);
   let ts = TimestampNanosecondArray::from(vec![

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-16 Thread via GitHub 


Dandandan commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4466606277

   > > Found by a `cargo fuzz` target 
(`fuzz/fuzz_targets/sql_physical_plan.rs`) seeded with SQL extracted from 
`datafusion/sqllogictest/test_files/`. The fuzzer triggered it from a mutated 
`generate_series` example by replacing a small numeric literal with 
`9223372036854775807` (`i64::MAX`).
   > 
   > Was this a general-purpose fuzzer that mutates raw bytes, with SLT queries 
used as seeds?
   
   Yeah raw bytes with SLT queries as seeds, indeed. Also it uses the generated 
binary AFAIK to find tokens / branches.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-16 Thread via GitHub 


Dandandan commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4466604103

   > Thanks [@Dandandan](https://github.com/Dandandan) I also came up 
DataFusion is prone to overflows in 
[#22137](https://github.com/apache/datafusion/issues/22137)
   > 
   > Btw, what tool did you use to test this?
   
   This issue was using https://github.com/rust-fuzz/cargo-fuzz + Claude to 
come up with the code / handling the reports.
   
   The further reports come from using `/goal` and asking to produce 20 unique 
reproducable crashes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-15 Thread via GitHub 


2010YOUY01 commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4465239915

   > Found by a `cargo fuzz` target (`fuzz/fuzz_targets/sql_physical_plan.rs`) 
seeded with SQL extracted from `datafusion/sqllogictest/test_files/`. The 
fuzzer triggered it from a mutated `generate_series` example by replacing a 
small numeric literal with `9223372036854775807` (`i64::MAX`).
   
   Was this a general-purpose fuzzer that mutates raw bytes, with SLT queries 
used as seeds?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [I] panic: capacity overflow in generate_series / range with large i64 arguments [datafusion]

2026-05-15 Thread via GitHub 


comphead commented on issue #22188:
URL: https://github.com/apache/datafusion/issues/22188#issuecomment-4464417035

   Thanks @Dandandan I also came up DataFusion is prone to overflows in #22137 
   
   Btw, what tool did you use to test this? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]