This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/rand_distr-0.6.0
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit 5382b1d983 chore(deps): bump rand_distr from 0.5.1 to 0.6.0
add d692df0358 feat: Optimize hash util for `MapArray` (#20179)
add c0e80edc30 perf: Optimize scalar fast path of atan2 (#20336)
add 726d73067f chore(deps): bump taiki-e/install-action from 2.67.27 to
2.68.0 (#20398)
add 5bfcf9500c feat: Implement Spark `bitmap_bit_position` function
(#20275)
add 08c09db5e5 feat: support sqllogictest output coloring (#20368)
add c699361cb4 fix: Handle Utf8View and LargeUtf8 separators in concat_ws
(#20361)
add 4f4e814ce3 perf: Optimize concat()/concat_ws() UDFs (#20317)
add c3f080774c perf: Optimize translate() UDF for scalar inputs (#20305)
add 0022d8e503 chore: Cleanup returning null arrays (#20423)
add 0294a22cf9 perf: Optimize `array_has()` for scalar needle (#20374)
add 3157a2e131 chore: fix labeler for `datafusion-functions-nested`
(#20442)
add b7f6090874 perf: Optimize lpad, rpad for ASCII strings (#20278)
add ba267acf8f build: update Rust toolchain version from 1.92.0 to 1.93.0
in `rust-toolchain.toml` (#20309)
add ace9cd44b7 perf: Optimize trim UDFs for single-character trims (#20328)
add 1ee782f783 Migrate Python usage to uv workspace (#20414)
add 0f7a405b8c feat: support Spark-compatible `json_tuple` function
(#20412)
add a936d0de95 test: Extend Spark Array functions: `array_repeat `,
`shuffle` and `slice` test coverage (#20420)
add 7f99947390 chore: Cleanup "!is_valid(i)" -> "is_null(i)" (#20453)
add fc98d5c282 feat: Implement Spark `bitmap_bucket_number` function
(#20288)
add 1736fd2a40 refactor: Extract sort-merge join filter logic into
separate module (#19614)
add 0d63ced04a Implement FFI table provider factory (#20326)
add 42dd4279de bench: Add criterion benchmark for sort merge join (#20464)
add d03601547a chore: group minor dependencies into single PR (#20457)
add d2c5666f5a chore(deps): bump taiki-e/install-action from 2.68.0 to
2.68.6 (#20467)
add 626bc01b04 chore(deps): bump astral-sh/setup-uv from 6.1.0 to 7.3.0
(#20468)
add 043f908b60 chore(deps): bump the all-other-cargo-deps group with 6
updates (#20470)
add cfdd7c180c chore(deps): bump testcontainers-modules from 0.14.0 to
0.15.0 (#20471)
add f488a9071b perf: Optimize scalar fast path for `regexp_like` and
rejects g inside combined flags like ig (#20354)
add c1ad8636a0 [Minor] Use buffer_unordered (#20462)
add bfc012e638 bench: Add IN list benchmarks for non-constant list
expressions (#20444)
add 9660c98743 perf: Use zero-copy slice instead of take kernel in sort
merge join (#20463)
add 7815732f0f feat(memory-tracking): implement arrow_buffer::MemoryPool
for MemoryPool (#18928)
add 60457d0b0a Runs-on for more actions (#20274)
add 89a8576171 docs: Document that adding new optimizer rules are
expensive (#20348)
add ed0323a2bb feat: support `arrays_zip` function (#20440)
add df8f818b29 chore: Avoid build fails on MinIO rate limits (#20472)
add d303f5817f chore: Add end-to-end benchmark for array_agg, code cleanup
(#20496)
add b9328b9734 Upgrade to sqlparser 0.61.0 (#20177)
add 7602913b0f Switch to the latest Mac OS (#20510)
add b6d46a6382 perf: Optimize `initcap()` (#20352)
add d59cdfe999 Fix name tracker (#19856)
add 11ef486e6c Runs-on for extended CI checks (#20511)
add 3aa34b33f5 chore(deps): bump strum from 0.27.2 to 0.28.0 (#20520)
add 4c0a6531ca chore(deps): bump taiki-e/install-action from 2.68.6 to
2.68.8 (#20518)
add 6c793694e9 chore(deps): bump the all-other-cargo-deps group with 2
updates (#20519)
add 0dfa542201 fix: HashJoin panic with dictionary-encoded columns in
multi-key joins (#20441)
add 4a41587bdf Make `custom_file_casts` example schema nullable to allow
null `id` values during casting (#20486)
add a9c090141d Add support for FFI config extensions (#19469)
add 9c85ac608f perf: Fix quadratic behavior of `to_array_of_size` (#20459)
add 17d770d6e5 fix: handle out of range errors in DATE_BIN instead of
panicking (#20221)
add 670dbf481c fix: prevent duplicate alias collision with user-provided
__datafusion_extracted names (#20432)
add e71e7a39bf chore: Cleanup code to use `repeat_n` in a few places
(#20527)
add 932418b20c chore(deps): bump strum_macros from 0.27.2 to 0.28.0
(#20521)
add db5197b742 chore: Replace `matches!` on fieldless enums with `==`
(#20525)
add b16ad9badc fix: SortMergeJoin don't wait for all input before emitting
(#20482)
add fdd36d0d21 Update comments on OptimizerRule about function name
matching (#20346)
add e80694e369 Remove recursive const check in `simplify_const_expr`
(#20234)
add b8cebdde2a Fix incorrect regex pattern in regex_replace_posix_groups
(#19827)
add 34dad2ccee Cache `PlanProperties`, add fast-path for
`with_new_children` (#19792)
add 585bbf35d3 perf: Optimize `array_has_any()` with scalar arg (#20385)
add 387e20cc58 Improve `HashJoinExecBuilder` to save state from previous
fields (#20276)
add 2347306943 [Minor] Fix error messages for `shrink` and `try_shrink`
(#20422)
add d75fcb83e3 Fix physical expr adapter to resolve physical fields by
name, not column index (#20485)
add e937cadbcc [fix] Add type coercion from NULL to Interval to make
date_bin more postgres compatible (#20499)
add d7d646164d feat: Implement Spark `bin` function (#20479)
add e6849945bf fix: `cardinality()` of an empty array should be zero
(#20533)
add e894a03bea perf: Use Hashbrown for array_distinct (#20538)
add 3a970c58ff Clamp early aggregation emit to the sort boundary when
using partial group ordering (#20446)
add 33b86fe02e perf: Cache num_output_rows in sort merge join to avoid
O(n) recount (#20478)
add bcd42b090a fix: Unaccounted spill sort in row_hash (#20314)
add a026e7da2f perf: Optimize heap handling in TopK operator (#20556)
add d6fb3608b0 perf: Optimize `array_position` for scalar needle (#20532)
add e76f0eebe3 fix: IS NULL panic with invalid function without input
arguments (#20306)
add 3ab1301c53 fix: handle empty delimiter in split_part (closes #20503)
(#20542)
add a257c29c26 add redirect for old upgrading.html URL to fix broken
changelog links (#20582)
add a79e6e6b39 fix(substrait): Correctly parse field references in
subqueries (#20439)
add bc600b3090 Split `push_down_filter.slt` into standalone sqllogictest
files to reduce long-tail runtime (#20566)
add e583fe9d22 Add deterministic per-file timing summary to sqllogictest
runner (#20569)
add c63ca330d7 fix: increase ROUND decimal precision to prevent overflow
truncation (#19926)
add 7ef62b988d chore: Enable workspace lint for all workspace members
(#20577)
add 451c79fc00 fix: Fix `array_to_string` with columnar third arg (#20536)
add e567cb91f4 Fix serde of window lead/lag defaults (#20608)
add 5d8249ff16 fix: Fix and Refactor Spark `shuffle` function (#20484)
add acec058cb5 perf: Use Arrow vectorized eq kernel for IN list with
column references (#20528)
add 73fbd48070 Upgrade DataFusion to arrow-rs/parquet 58.0.0 /
`object_store` 0.13.0 (#19728)
add be9d9625e4 chore(deps): bump rand_distr from 0.5.1 to 0.6.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5382b1d983)
\
N -- N -- N refs/heads/dependabot/cargo/main/rand_distr-0.6.0
(be9d9625e4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/dependabot.yml | 17 +
.github/workflows/audit.yml | 2 +-
.github/workflows/docs.yaml | 15 +-
.github/workflows/docs_pr.yaml | 15 +-
.github/workflows/extended.yml | 22 +-
.github/workflows/labeler/labeler-config.yml | 2 +-
.github/workflows/rust.yml | 47 +-
Cargo.lock | 370 ++++---
Cargo.toml | 26 +-
benchmarks/Cargo.toml | 2 +-
benchmarks/bench.sh | 143 +--
benchmarks/pyproject.toml | 6 +
benchmarks/requirements.txt | 18 -
datafusion-cli/Cargo.toml | 7 +-
datafusion-cli/src/exec.rs | 10 +-
datafusion-cli/src/functions.rs | 8 +-
datafusion-cli/src/highlighter.rs | 3 +-
datafusion-cli/src/main.rs | 12 +-
datafusion-cli/src/object_storage.rs | 25 +-
datafusion-cli/src/object_storage/instrumented.rs | 128 ++-
datafusion-cli/src/print_format.rs | 20 +-
datafusion-cli/src/print_options.rs | 16 +-
datafusion-cli/tests/cli_integration.rs | 57 +-
.../custom_data_source/adapter_serialization.rs | 2 +-
.../examples/custom_data_source/csv_json_opener.rs | 2 +-
.../custom_data_source/custom_datasource.rs | 6 +-
.../custom_data_source/custom_file_casts.rs | 6 +-
.../custom_data_source/default_column_values.rs | 4 +-
.../examples/data_io/json_shredding.rs | 4 +-
.../examples/data_io/parquet_advanced_index.rs | 8 +-
.../memory_pool_execution_plan.rs | 4 +-
.../ffi/ffi_example_table_provider/Cargo.toml | 3 +
.../examples/ffi/ffi_module_interface/Cargo.toml | 3 +
.../examples/ffi/ffi_module_loader/Cargo.toml | 3 +
.../examples/proto/composed_extension_codec.rs | 4 +-
.../examples/relation_planner/table_sample.rs | 6 +-
datafusion/catalog/src/memory/table.rs | 6 +-
datafusion/common/Cargo.toml | 4 +
datafusion/common/benches/scalar_to_array.rs | 107 ++
datafusion/common/benches/with_hashes.rs | 252 ++++-
datafusion/common/src/alias.rs | 10 +
datafusion/common/src/config.rs | 35 +-
datafusion/common/src/file_options/mod.rs | 2 +-
.../common/src/file_options/parquet_writer.rs | 8 +-
datafusion/common/src/hash_utils.rs | 126 ++-
datafusion/common/src/scalar/mod.rs | 98 +-
datafusion/common/src/stats.rs | 2 +-
datafusion/common/src/types/native.rs | 4 +-
datafusion/core/benches/aggregate_query_sql.rs | 33 +
datafusion/core/benches/data_utils/mod.rs | 101 +-
datafusion/core/benches/parquet_query_sql.rs | 4 +-
datafusion/core/benches/parquet_struct_query.rs | 4 +-
datafusion/core/benches/push_down_filter.rs | 2 +-
datafusion/core/benches/reset_plan_states.rs | 2 +
datafusion/core/benches/spm.rs | 5 +-
datafusion/core/benches/sql_query_with_io.rs | 2 +-
datafusion/core/src/datasource/file_format/csv.rs | 29 +-
.../core/src/datasource/file_format/parquet.rs | 38 +-
.../core/src/datasource/physical_plan/csv.rs | 2 +-
datafusion/core/src/physical_planner.rs | 26 +-
datafusion/core/src/test/object_store.rs | 65 +-
.../tests/catalog_listing/pruned_partition_list.rs | 2 +-
datafusion/core/tests/custom_sources_cases/mod.rs | 9 +-
.../provider_filter_pushdown.rs | 9 +-
.../core/tests/custom_sources_cases/statistics.rs | 6 +-
datafusion/core/tests/dataframe/mod.rs | 9 +-
.../core/tests/datasource/object_store_access.rs | 113 +-
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/once_exec.rs | 6 +-
datafusion/core/tests/fuzz_cases/pruning.rs | 4 +-
.../spilling_fuzz_in_memory_constrained_env.rs | 16 +-
.../core/tests/fuzz_cases/topk_filter_pushdown.rs | 2 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 2 +-
datafusion/core/tests/parquet/custom_reader.rs | 2 +-
datafusion/core/tests/parquet/expr_adapter.rs | 2 +-
.../core/tests/parquet/external_access_plan.rs | 2 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 4 +-
datafusion/core/tests/parquet/mod.rs | 2 +-
.../physical_optimizer/enforce_distribution.rs | 6 +-
.../tests/physical_optimizer/join_selection.rs | 12 +-
.../tests/physical_optimizer/pushdown_utils.rs | 2 +-
.../replace_with_order_preserving_variants.rs | 2 +-
.../core/tests/physical_optimizer/test_utils.rs | 8 +-
datafusion/core/tests/sql/path_partition.rs | 65 +-
.../core/tests/tracing/traceable_object_store.rs | 34 +-
.../core/tests/user_defined/insert_operation.rs | 20 +-
.../core/tests/user_defined/user_defined_plan.rs | 10 +-
datafusion/datasource-arrow/src/file_format.rs | 3 +-
datafusion/datasource-arrow/src/source.rs | 2 +-
datafusion/datasource-avro/src/file_format.rs | 2 +-
datafusion/datasource-avro/src/source.rs | 2 +-
datafusion/datasource-csv/src/file_format.rs | 4 +-
datafusion/datasource-json/src/file_format.rs | 2 +-
datafusion/datasource-json/src/source.rs | 2 +-
.../benches/parquet_nested_filter_pushdown.rs | 12 +-
datafusion/datasource-parquet/src/file_format.rs | 20 +-
datafusion/datasource-parquet/src/opener.rs | 11 +-
.../datasource-parquet/src/row_group_filter.rs | 3 +-
datafusion/datasource/Cargo.toml | 2 +-
datafusion/datasource/src/mod.rs | 2 +-
datafusion/datasource/src/sink.rs | 6 +-
datafusion/datasource/src/source.rs | 24 +-
datafusion/datasource/src/url.rs | 42 +-
datafusion/execution/Cargo.toml | 4 +
datafusion/execution/src/memory_pool/arrow.rs | 142 +++
datafusion/execution/src/memory_pool/mod.rs | 11 +-
datafusion/expr-common/src/signature.rs | 2 +-
datafusion/expr-common/src/type_coercion/binary.rs | 2 +-
datafusion/expr/src/logical_plan/plan.rs | 2 +-
datafusion/expr/src/type_coercion/functions.rs | 2 +-
datafusion/ffi/src/config/extension_options.rs | 288 +++++
datafusion/ffi/src/config/mod.rs | 169 +++
datafusion/ffi/src/execution_plan.rs | 20 +-
datafusion/ffi/src/lib.rs | 2 +
datafusion/ffi/src/session/config.rs | 39 +-
datafusion/ffi/src/table_provider_factory.rs | 429 ++++++++
datafusion/ffi/src/tests/async_provider.rs | 8 +-
datafusion/ffi/src/tests/config.rs | 51 +
datafusion/ffi/src/tests/mod.rs | 25 +-
datafusion/ffi/src/tests/table_provider_factory.rs | 58 +
datafusion/ffi/tests/ffi_config.rs | 124 +++
datafusion/ffi/tests/ffi_integration.rs | 46 +-
datafusion/functions-aggregate/src/regr.rs | 2 +-
datafusion/functions-nested/Cargo.toml | 5 +
datafusion/functions-nested/benches/array_has.rs | 851 +++++++++++----
.../functions-nested/benches/array_position.rs | 237 ++++
datafusion/functions-nested/src/array_has.rs | 297 ++++-
datafusion/functions-nested/src/arrays_zip.rs | 336 ++++++
datafusion/functions-nested/src/cardinality.rs | 13 +-
datafusion/functions-nested/src/lib.rs | 3 +
datafusion/functions-nested/src/position.rs | 163 ++-
datafusion/functions-nested/src/range.rs | 25 +-
datafusion/functions-nested/src/set_ops.rs | 2 +-
datafusion/functions-nested/src/string.rs | 49 +-
datafusion/functions-window/src/nth_value.rs | 2 +-
datafusion/functions/Cargo.toml | 5 +
datafusion/functions/benches/{cot.rs => atan2.rs} | 81 +-
datafusion/functions/benches/initcap.rs | 108 +-
datafusion/functions/benches/pad.rs | 182 +++-
datafusion/functions/benches/regx.rs | 42 +-
datafusion/functions/benches/translate.rs | 48 +-
datafusion/functions/benches/trim.rs | 132 +++
datafusion/functions/src/core/getfield.rs | 2 +-
datafusion/functions/src/datetime/date_bin.rs | 291 +++--
datafusion/functions/src/macros.rs | 102 +-
datafusion/functions/src/math/gcd.rs | 7 +-
datafusion/functions/src/math/round.rs | 472 +++++++-
datafusion/functions/src/regex/regexplike.rs | 239 +++-
datafusion/functions/src/regex/regexpreplace.rs | 48 +-
datafusion/functions/src/string/btrim.rs | 6 +-
datafusion/functions/src/string/common.rs | 82 +-
datafusion/functions/src/string/concat.rs | 15 +-
datafusion/functions/src/string/concat_ws.rs | 182 +++-
datafusion/functions/src/string/ltrim.rs | 6 +-
datafusion/functions/src/string/rtrim.rs | 6 +-
datafusion/functions/src/string/split_part.rs | 130 ++-
datafusion/functions/src/strings.rs | 23 +-
datafusion/functions/src/unicode/find_in_set.rs | 26 +-
datafusion/functions/src/unicode/initcap.rs | 197 +++-
datafusion/functions/src/unicode/lpad.rs | 104 +-
datafusion/functions/src/unicode/rpad.rs | 112 +-
datafusion/functions/src/unicode/translate.rs | 186 +++-
datafusion/macros/Cargo.toml | 2 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 4 -
.../src/decorrelate_predicate_subquery.rs | 2 +-
.../optimizer/src/extract_leaf_expressions.rs | 41 +-
datafusion/optimizer/src/optimizer.rs | 55 +-
datafusion/optimizer/src/push_down_filter.rs | 2 +-
datafusion/physical-expr-adapter/Cargo.toml | 3 +
.../physical-expr-adapter/src/schema_rewriter.rs | 83 +-
.../physical-expr-common/src/binary_view_map.rs | 2 +-
datafusion/physical-expr-common/src/datum.rs | 4 +-
datafusion/physical-expr/benches/in_list.rs | 166 +++
.../src/equivalence/properties/mod.rs | 7 +-
datafusion/physical-expr/src/expressions/case.rs | 2 +-
datafusion/physical-expr/src/expressions/cast.rs | 4 +-
.../physical-expr/src/expressions/in_list.rs | 245 ++++-
.../physical-expr/src/expressions/try_cast.rs | 4 +-
datafusion/physical-expr/src/partitioning.rs | 2 +-
.../src/simplifier/const_evaluator.rs | 102 +-
datafusion/physical-expr/src/simplifier/mod.rs | 9 +-
datafusion/physical-expr/src/simplifier/not.rs | 4 +
.../src/combine_partial_final_agg.rs | 2 +-
.../physical-optimizer/src/enforce_distribution.rs | 85 +-
datafusion/physical-optimizer/src/ensure_coop.rs | 6 +-
.../physical-optimizer/src/join_selection.rs | 11 +-
datafusion/physical-optimizer/src/optimizer.rs | 6 +
.../physical-optimizer/src/output_requirements.rs | 6 +-
datafusion/physical-plan/Cargo.toml | 5 +
.../physical-plan/benches/sort_merge_join.rs | 204 ++++
datafusion/physical-plan/src/aggregates/mod.rs | 179 ++-
.../physical-plan/src/aggregates/row_hash.rs | 171 ++-
datafusion/physical-plan/src/analyze.rs | 6 +-
datafusion/physical-plan/src/async_func.rs | 27 +-
datafusion/physical-plan/src/buffer.rs | 23 +-
datafusion/physical-plan/src/coalesce_batches.rs | 25 +-
.../physical-plan/src/coalesce_partitions.rs | 26 +-
datafusion/physical-plan/src/coop.rs | 24 +-
datafusion/physical-plan/src/display.rs | 2 +-
datafusion/physical-plan/src/empty.rs | 8 +-
datafusion/physical-plan/src/execution_plan.rs | 74 +-
datafusion/physical-plan/src/explain.rs | 6 +-
datafusion/physical-plan/src/filter.rs | 31 +-
datafusion/physical-plan/src/joins/cross_join.rs | 28 +-
.../physical-plan/src/joins/hash_join/exec.rs | 400 +++----
.../src/joins/hash_join/inlist_builder.rs | 58 +-
.../physical-plan/src/joins/nested_loop_join.rs | 36 +-
.../src/joins/piecewise_merge_join/classic_join.rs | 2 +-
.../src/joins/piecewise_merge_join/exec.rs | 90 +-
.../src/joins/sort_merge_join/exec.rs | 27 +-
.../src/joins/sort_merge_join/filter.rs | 595 ++++++++++
.../physical-plan/src/joins/sort_merge_join/mod.rs | 1 +
.../src/joins/sort_merge_join/stream.rs | 660 +++--------
.../src/joins/sort_merge_join/tests.rs | 519 ++++++++-
.../physical-plan/src/joins/symmetric_hash_join.rs | 22 +-
datafusion/physical-plan/src/joins/utils.rs | 2 +-
datafusion/physical-plan/src/limit.rs | 47 +-
datafusion/physical-plan/src/memory.rs | 19 +-
datafusion/physical-plan/src/placeholder_row.rs | 8 +-
datafusion/physical-plan/src/projection.rs | 20 +-
datafusion/physical-plan/src/recursive_query.rs | 6 +-
datafusion/physical-plan/src/repartition/mod.rs | 32 +-
datafusion/physical-plan/src/sorts/mod.rs | 2 +
datafusion/physical-plan/src/sorts/partial_sort.rs | 27 +-
datafusion/physical-plan/src/sorts/sort.rs | 102 +-
.../src/sorts/sort_preserving_merge.rs | 36 +-
datafusion/physical-plan/src/sorts/stream.rs | 164 ++-
datafusion/physical-plan/src/spill/mod.rs | 14 +-
.../physical-plan/src/spill/spill_manager.rs | 51 +-
datafusion/physical-plan/src/streaming.rs | 8 +-
datafusion/physical-plan/src/test.rs | 14 +-
datafusion/physical-plan/src/test/exec.rs | 149 ++-
datafusion/physical-plan/src/topk/mod.rs | 44 +-
datafusion/physical-plan/src/union.rs | 41 +-
datafusion/physical-plan/src/unnest.rs | 24 +-
.../src/windows/bounded_window_agg_exec.rs | 20 +-
datafusion/physical-plan/src/windows/mod.rs | 12 +
.../physical-plan/src/windows/window_agg_exec.rs | 24 +-
datafusion/physical-plan/src/work_table.rs | 8 +-
datafusion/proto/src/physical_plan/to_proto.rs | 3 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 45 +
datafusion/pruning/src/pruning_predicate.rs | 2 +-
datafusion/spark/Cargo.toml | 1 +
datafusion/spark/src/function/array/shuffle.rs | 16 +-
.../src/function/bitmap/bitmap_bit_position.rs | 141 +++
.../src/function/bitmap/bitmap_bucket_number.rs | 141 +++
datafusion/spark/src/function/bitmap/mod.rs | 23 +-
datafusion/spark/src/function/datetime/last_day.rs | 6 +-
datafusion/spark/src/function/datetime/next_day.rs | 9 +-
datafusion/spark/src/function/hash/sha2.rs | 9 +-
datafusion/spark/src/function/json/json_tuple.rs | 244 +++++
datafusion/spark/src/function/json/mod.rs | 17 +-
.../spark/src/function/map/map_from_arrays.rs | 4 +-
datafusion/spark/src/function/map/utils.rs | 4 +-
datafusion/spark/src/function/math/bin.rs | 109 ++
datafusion/spark/src/function/math/mod.rs | 8 +
datafusion/spark/src/function/math/width_bucket.rs | 4 +-
.../spark/src/function/string/format_string.rs | 2 +-
datafusion/sql/src/expr/function.rs | 2 +-
datafusion/sql/src/expr/mod.rs | 6 +
datafusion/sql/src/planner.rs | 8 +-
datafusion/sql/src/select.rs | 1 +
datafusion/sql/src/statement.rs | 117 +-
datafusion/sql/src/unparser/ast.rs | 7 +-
datafusion/sql/src/unparser/dialect.rs | 1 +
datafusion/sql/src/unparser/expr.rs | 7 +
datafusion/sql/src/utils.rs | 2 +
datafusion/sqllogictest/Cargo.toml | 2 +-
datafusion/sqllogictest/README.md | 30 +
datafusion/sqllogictest/bin/sqllogictests.rs | 274 ++++-
.../src/engines/datafusion_engine/normalize.rs | 2 +-
datafusion/sqllogictest/test_files/array.slt | 339 +++++-
.../sqllogictest/test_files/date_bin_errors.slt | 60 +
.../test_files/datetime/timestamps.slt | 12 +
datafusion/sqllogictest/test_files/decimal.slt | 2 +-
.../test_files/dynamic_filter_pushdown_config.slt | 2 +-
.../sqllogictest/test_files/explain_tree.slt | 14 +-
datafusion/sqllogictest/test_files/expr.slt | 39 +
datafusion/sqllogictest/test_files/functions.slt | 14 +
datafusion/sqllogictest/test_files/joins.slt | 28 +
datafusion/sqllogictest/test_files/limit.slt | 8 +-
.../test_files/projection_pushdown.slt | 41 +
.../sqllogictest/test_files/push_down_filter.slt | 745 -------------
.../test_files/push_down_filter_outer_joins.slt | 264 +++++
.../test_files/push_down_filter_parquet.slt | 188 ++++
.../test_files/push_down_filter_regression.slt | 200 ++++
.../test_files/push_down_filter_unnest.slt | 148 +++
datafusion/sqllogictest/test_files/scalar.slt | 68 +-
datafusion/sqllogictest/test_files/select.slt | 27 +-
.../test_files/spark/array/array_repeat.slt | 20 +
.../test_files/spark/array/shuffle.slt | 30 +
.../sqllogictest/test_files/spark/array/slice.slt | 15 +
.../spark/bitmap/bitmap_bit_position.slt | 112 ++
.../spark/bitmap/bitmap_bucket_number.slt | 122 +++
.../test_files/spark/datetime/make_interval.slt | 10 +-
.../test_files/spark/json/json_tuple.slt | 154 +++
.../sqllogictest/test_files/spark/math/bin.slt | 79 +-
.../sqllogictest/test_files/type_coercion.slt | 21 +
datafusion/sqllogictest/test_files/window.slt | 8 +-
datafusion/substrait/Cargo.toml | 1 -
.../logical_plan/consumer/expr/field_reference.rs | 53 +-
.../src/logical_plan/consumer/expr/mod.rs | 5 +-
.../src/logical_plan/consumer/expr/subquery.rs | 44 +-
.../substrait/src/logical_plan/consumer/plan.rs | 5 +-
.../src/logical_plan/consumer/rel/exchange_rel.rs | 3 +-
.../src/logical_plan/consumer/rel/project_rel.rs | 15 +-
.../logical_plan/consumer/substrait_consumer.rs | 123 ++-
.../substrait/src/logical_plan/consumer/utils.rs | 224 +++-
.../src/logical_plan/producer/expr/cast.rs | 2 +-
.../substrait/tests/cases/consumer_integration.rs | 118 +-
datafusion/substrait/tests/cases/logical_plans.rs | 35 +-
.../tests/cases/roundtrip_logical_plan.rs | 65 +-
.../nested_correlated_subquery.substrait.json | 265 +++++
datafusion/wasmtest/Cargo.toml | 2 +-
datafusion/wasmtest/src/lib.rs | 2 +-
dev/pyproject.toml | 5 +
dev/release/README.md | 6 +-
dev/requirements.txt | 2 -
dev/update_arrow_deps.py | 2 +-
dev/update_datafusion_versions.py | 2 +-
docs/README.md | 15 +-
docs/pyproject.toml | 13 +
docs/requirements.txt | 24 -
docs/source/conf.py | 1 +
docs/source/contributor-guide/index.md | 28 +-
docs/source/contributor-guide/testing.md | 4 +-
docs/source/download.md | 2 +-
.../library-user-guide/custom-table-providers.md | 6 +-
docs/source/library-user-guide/upgrading/53.0.0.md | 74 ++
docs/source/user-guide/crate-configuration.md | 2 +-
docs/source/user-guide/example-usage.md | 2 +-
docs/source/user-guide/sql/scalar_functions.md | 77 +-
pyproject.toml | 2 +
rust-toolchain.toml | 2 +-
uv.lock | 1149 ++++++++++++++++++++
335 files changed, 15915 insertions(+), 4190 deletions(-)
create mode 100644 benchmarks/pyproject.toml
delete mode 100644 benchmarks/requirements.txt
create mode 100644 datafusion/common/benches/scalar_to_array.rs
create mode 100644 datafusion/execution/src/memory_pool/arrow.rs
create mode 100644 datafusion/ffi/src/config/extension_options.rs
create mode 100644 datafusion/ffi/src/config/mod.rs
create mode 100644 datafusion/ffi/src/table_provider_factory.rs
create mode 100644 datafusion/ffi/src/tests/config.rs
create mode 100644 datafusion/ffi/src/tests/table_provider_factory.rs
create mode 100644 datafusion/ffi/tests/ffi_config.rs
create mode 100644 datafusion/functions-nested/benches/array_position.rs
create mode 100644 datafusion/functions-nested/src/arrays_zip.rs
copy datafusion/functions/benches/{cot.rs => atan2.rs} (63%)
create mode 100644 datafusion/physical-plan/benches/sort_merge_join.rs
create mode 100644 datafusion/physical-plan/src/joins/sort_merge_join/filter.rs
create mode 100644 datafusion/spark/src/function/bitmap/bitmap_bit_position.rs
create mode 100644 datafusion/spark/src/function/bitmap/bitmap_bucket_number.rs
create mode 100644 datafusion/spark/src/function/json/json_tuple.rs
create mode 100644 datafusion/spark/src/function/math/bin.rs
create mode 100644 datafusion/sqllogictest/test_files/date_bin_errors.slt
delete mode 100644 datafusion/sqllogictest/test_files/push_down_filter.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_outer_joins.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_parquet.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_regression.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_unnest.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/bitmap/bitmap_bit_position.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/bitmap/bitmap_bucket_number.slt
create mode 100644 datafusion/sqllogictest/test_files/spark/json/json_tuple.slt
create mode 100644
datafusion/substrait/tests/testdata/test_plans/nested_correlated_subquery.substrait.json
create mode 100644 dev/pyproject.toml
delete mode 100644 dev/requirements.txt
create mode 100644 docs/pyproject.toml
delete mode 100644 docs/requirements.txt
create mode 100644 pyproject.toml
create mode 100644 uv.lock
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
