2010YOUY01 opened a new issue, #27082: URL: https://github.com/apache/datafusion/issues/27082
Fuzz tests should have configurable time budgets, be easy to reproduce, and explore new seeds over time. Existing implementations often miss one or more of these properties. Adopt the centralized `FuzzTest` trait and driver from [#27048](https://github.com/apache/datafusion/pull/27048) to: - Make these conventions a required contract for every fuzzer. - Share repetition, budgets, seed reporting, and replay, keeping implementations simple. Spill pool will be migrated in the initial PR. The later plan is - Refactor other existing fuzzing UTs to use the centralized trait and runner - Require new fuzz tests follow this convention ## Tracker - [ ] Add documentation for fuzzing UTs and new trait The remaining 30 suite groups below cover 152 test definitions, grouped by source file. - [ ] **Aggregation** — `async fn test_min() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs#L63)). Also: `test_first_val`, `test_last_val`, `test_max`, `test_sum`, `test_count`, `test_median`, `streaming_aggregate_test`, `group_by_strings`. - [ ] **String DISTINCT counts** — `async fn distinct_count_string_test() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/distinct_count_string_fuzz.rs#L19)). - [ ] **Equivalence: ordering** — `fn test_ordering_satisfy_with_equivalence_random() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/equivalence/ordering.rs#L32)). Also: `test_ordering_satisfy_with_equivalence_complex_random`, `test_ordering_satisfy_with_casts_and_windows_random`. - [ ] **Equivalence: projection** — `fn project_orderings_random() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/equivalence/projection.rs#L25)). Also: `ordering_satisfy_after_projection_random`. - [ ] **Equivalence: properties** — `fn test_find_longest_permutation_random() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/equivalence/properties.rs#L26)). Also: `test_find_longest_permutation_with_casts_and_windows_random`. - [ ] **Joins** — `async fn test_inner_join_1k_filtered() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/join_fuzz.rs#L92)). Also: `test_inner_join_1k`, `test_left_join_1k`, `test_left_join_1k_filtered`, `test_right_join_1k`, `test_right_join_1k_filtered`, `test_full_join_1k`, `test_full_join_1k_filtered`, `test_left_semi_join_1k`, `test_left_semi_join_1k_filtered`, `test_right_semi_join_1k`, `test_right_semi_join_1k_filtered`, `test_left_anti_join_1k`, `test_left_anti_join_1k_filtered`, `test_right_anti_join_1k`, `test_right_anti_join_1k_filtered`, `test_left_mark_join_1k`, `test_left_mark_join_1k_filtered`, `test_right_mark_join_1k`, `test_right_mark_join_1k_filtered`, `test_inner_join_1k_binary_filtered`, `test_inner_join_1k_binary`, `test_left_join_1k_binary`, `test_left_join_1k_binary_filtered`, `test_right_join_1k_binary`, `test_right_join_1k_binary_filtered`, `test_full_join_1k_binary`, `test_full_join_1k_binary_filtered`, `test_left_semi_join_1k_binary`, `test_left_semi_join_1k_binary_filtered`, `test_right_semi_join_1k_binary`, `test_right_semi_join_1k_binary_filtered`, `test_left_anti_join_1k_binary`, `test_left_anti_join_1k_binary_filtered`, `test_right_anti_join_1k_binary`, `test_right_anti_join_1k_binary_filtered`, `test_left_mark_join_1k_binary`, `test_left_mark_join_1k_binary_filtered`, `test_right_mark_join_1k_binary`, `test_right_mark_join_1k_binary_filtered`, `test_inner_join_1k_low_cardinality`, `test_inner_join_1k_low_cardinality_filtered`, `test_left_join_1k_low_cardinality`, `test_left_join_1k_low_cardinality_filtered`, `test_right_join_1k_low_cardinality`, `test_right_join_1k_low_cardinality_filtered`, `test_full_join_1k_low_cardinality`, `test_full_join_1k_low_cardinality_filtered`, `test_filtered_join_spill_fuzz`, `test_nested_loop_join_spill_fuzz`, `fuzz_pwmj_matches_nested_loop`. - [ ] **Leaf-expression pushdown** — `async fn leaf_pushdown_fuzz() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/leaf_pushdown_fuzz.rs#L758)). Also: `leaf_pushdown_parquet_fuzz`, `leaf_pushdown_fuzz_extended`. - [ ] **Sort / Top-K limits** — `async fn test_sort_topk_i32() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/limit_fuzz.rs#L19)). Also: `test_sort_topk_f64`, `test_sort_topk_str`, `test_sort_topk_i64str`. - [ ] **Stream merging** — `async fn test_merge_2() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/merge_fuzz.rs#L25)). Also: `test_merge_2_no_overlap`, `test_merge_3`, `test_merge_3_gaps`. - [ ] **Null-aware joins** — `async fn null_aware_join_matches_reference() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/null_aware_join_fuzz.rs#L30)). - [ ] **UTF-8 pruning** — `async fn test_utf8_eq() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/pruning.rs#L33)). Also: `test_utf8_not_eq`, `test_utf8_lt`, `test_utf8_lt_eq`, `test_utf8_gt`, `test_utf8_gt_eq`, `test_utf8_like`, `test_utf8_not_like`, `test_utf8_like_prefix`, `test_utf8_like_suffix`, `test_utf8_not_like_prefix`, `test_utf8_not_like_ecsape`, `test_utf8_not_like_suffix`, `test_utf8_not_like_suffix_one`. - [ ] **ScalarValue conversions** — `fn scalar_value_eq_array_consistency() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/scalar_value_fuzz.rs#L18)). Also: `scalar_value_iter_to_array_roundtrip`, `scalar_value_iter_to_array_rejects_mixed_parameterized_types`, `scalar_value_to_array_roundtrip`. - [ ] **Sorting and spilling** — `async fn test_sort_10k_mem() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/sort_fuzz.rs#L30)). Also: `test_sort_100k_mem`, `test_sort_strings_100k_mem`, `test_sort_multi_columns_100k_mem`, `test_sort_unlimited_mem`. - [ ] **Sort-preserving repartition** — `async fn stream_merge_multi_order_preserve() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/sort_preserving_repartition_fuzz.rs#L202)). Also: `sort_preserving_repartition_test`. - [ ] **Sort queries** — `async fn sort_query_fuzzer_runner() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/sort_query_fuzz.rs#L36)). - [ ] **Top-K filter pushdown** — `async fn test_fuzz_topk_filter_pushdown() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/topk_filter_pushdown.rs#L269)). - [ ] **Window aggregation** — `async fn window_bounded_window_random_comparison() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/core/tests/fuzz_cases/window_fuzz.rs#L47)). - [ ] **Timestamp arithmetic** — `fn timestamp_op_random_tests() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/common/src/scalar/mod.rs#L10156)). - [ ] **Memory accounting** — `fn test_randomized_count_uncount_sequence() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/common/src/utils/memory.rs#L1491)). - [ ] **LRU cache** — `fn test_fuzzy() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/execution/src/cache/lru_queue.rs#L498)). - [ ] **Grouped accumulation** — `fn accumulate_fuzz() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/functions-aggregate-common/src/aggregate/groups_accumulator/accumulate.rs#L751)). - [ ] **Moving min/max** — `fn moving_min_tests() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/functions-aggregate/src/min_max.rs#L1386)). Also: `moving_max_tests`. - [ ] **List filtering** — `fn list_filter_matches_reference_on_random_lists() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/functions-nested/src/utils.rs#L710)). - [ ] **Boolean conjunctions** — `fn test_conjunction_matches_kleene_and() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-expr/src/expressions/binary.rs#L6627)). - [ ] **Unmarked join pairs** — `fn unmarked_pairs_match_brute_force() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-plan/src/joins/hash_join/not_in.rs#L1814)). - [ ] **Routed hash lookups** — `fn routed_lookup_fuzz() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs#L1207)). - [ ] **Key-range bitmap union** — `fn union_fuzz() {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-plan/src/joins/key_range_bitmap.rs#L599)). - [ ] **Partitioned Top-K** — `async fn test_partitioned_topk_matches_bruteforce() -> Result<()> {` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-plan/src/topk/mod.rs#L5142)). Also: `test_partitioned_topk_store_bookkeeping_holds_under_stress`, `test_partitioned_topk_dense_rank_matches_bruteforce`, `test_partitioned_topk_dense_rank_contents_bytes_tracks_recompute`, `test_partitioned_topk_rank_bookkeeping_tracks_recompute`. - [ ] **Symmetric hash joins** — `async fn complex_join_all_one_ascending_numeric(` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-plan/src/joins/symmetric_hash_join.rs#L2731)). Also: `join_all_one_ascending_numeric`, `join_without_sort_information`, `join_without_filter`, `join_all_one_descending_numeric_particular`, `testing_with_temporal_columns`, `test_with_interval_columns`, `testing_ascending_float_pruning`. - [ ] **Interval constraint propagation** — `integer_float_case_1!(case_1_i32, generate_case_i32, i32, Int32);` ([source](https://github.com/apache/datafusion/blob/4591113d2a49fcd7413b3eb6894bd9f031296d1c/datafusion/physical-expr/src/intervals/cp_solver.rs#L1138)). Also: `case_1_i64`, `case_1_f64`, `case_1_f32`, `case_2_i32`, `case_2_i64`, `case_2_f64`, `case_2_f32`, `case_3_i32`, `case_3_i64`, `case_3_f64`, `case_3_f32`, `case_4_i32`, `case_4_i64`, `case_4_f64`, `case_4_f32`, `case_5_i32`, `case_5_i64`, `case_5_f64`, `case_5_f32`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
