2010YOUY01 commented on issue #26700: URL: https://github.com/apache/datafusion/issues/26700#issuecomment-6382028296
Thanks for driving this! IMO, this is the highest-ROI improvement we can make to our testing right now. ### Suggestion: make it more automatic I think the existing practice is manual fuzzing: - per query: set different `batch_size` values and repeat the same query in an `slt` file - per file: config matrix I think it would be great if we could make this more automatic (no need to manually define which `batch_size` values to test). The implementation might look like: - For common configs like `batch_size`/`target_partitions`, let the SLT runner decide the values (Q1: batch_size=8192, Q2: batch_size=7, ...) - For feature-specific flags, keep using per-file/per-query config matrices - Add a mechanism to skip EXPLAIN-like tests Then the runner could generate random configs across different fuzzing runs, and we could set the CI runner to run as many tests as possible within a time limit, instead of enumerating all combinations, which could become very slow as we keep adding config matrix dimensions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
