This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/getrandom-0.4.1
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit af6f730837 chore(deps): bump getrandom from 0.3.4 to 0.4.1
add 0dfa542201 fix: HashJoin panic with dictionary-encoded columns in
multi-key joins (#20441)
add 4a41587bdf Make `custom_file_casts` example schema nullable to allow
null `id` values during casting (#20486)
add a9c090141d Add support for FFI config extensions (#19469)
add 9c85ac608f perf: Fix quadratic behavior of `to_array_of_size` (#20459)
add 17d770d6e5 fix: handle out of range errors in DATE_BIN instead of
panicking (#20221)
add 670dbf481c fix: prevent duplicate alias collision with user-provided
__datafusion_extracted names (#20432)
add e71e7a39bf chore: Cleanup code to use `repeat_n` in a few places
(#20527)
add 932418b20c chore(deps): bump strum_macros from 0.27.2 to 0.28.0
(#20521)
add db5197b742 chore: Replace `matches!` on fieldless enums with `==`
(#20525)
add b16ad9badc fix: SortMergeJoin don't wait for all input before emitting
(#20482)
add fdd36d0d21 Update comments on OptimizerRule about function name
matching (#20346)
add e80694e369 Remove recursive const check in `simplify_const_expr`
(#20234)
add b8cebdde2a Fix incorrect regex pattern in regex_replace_posix_groups
(#19827)
add 34dad2ccee Cache `PlanProperties`, add fast-path for
`with_new_children` (#19792)
add 585bbf35d3 perf: Optimize `array_has_any()` with scalar arg (#20385)
add 387e20cc58 Improve `HashJoinExecBuilder` to save state from previous
fields (#20276)
add 2347306943 [Minor] Fix error messages for `shrink` and `try_shrink`
(#20422)
add d75fcb83e3 Fix physical expr adapter to resolve physical fields by
name, not column index (#20485)
add e937cadbcc [fix] Add type coercion from NULL to Interval to make
date_bin more postgres compatible (#20499)
add d7d646164d feat: Implement Spark `bin` function (#20479)
add e6849945bf fix: `cardinality()` of an empty array should be zero
(#20533)
add e894a03bea perf: Use Hashbrown for array_distinct (#20538)
add 3a970c58ff Clamp early aggregation emit to the sort boundary when
using partial group ordering (#20446)
add 33b86fe02e perf: Cache num_output_rows in sort merge join to avoid
O(n) recount (#20478)
add bcd42b090a fix: Unaccounted spill sort in row_hash (#20314)
add a026e7da2f perf: Optimize heap handling in TopK operator (#20556)
add d6fb3608b0 perf: Optimize `array_position` for scalar needle (#20532)
add e76f0eebe3 fix: IS NULL panic with invalid function without input
arguments (#20306)
add 3ab1301c53 fix: handle empty delimiter in split_part (closes #20503)
(#20542)
add a257c29c26 add redirect for old upgrading.html URL to fix broken
changelog links (#20582)
add a79e6e6b39 fix(substrait): Correctly parse field references in
subqueries (#20439)
add bc600b3090 Split `push_down_filter.slt` into standalone sqllogictest
files to reduce long-tail runtime (#20566)
add e583fe9d22 Add deterministic per-file timing summary to sqllogictest
runner (#20569)
add c63ca330d7 fix: increase ROUND decimal precision to prevent overflow
truncation (#19926)
add 7ef62b988d chore: Enable workspace lint for all workspace members
(#20577)
add 451c79fc00 fix: Fix `array_to_string` with columnar third arg (#20536)
add e567cb91f4 Fix serde of window lead/lag defaults (#20608)
add 5d8249ff16 fix: Fix and Refactor Spark `shuffle` function (#20484)
add acec058cb5 perf: Use Arrow vectorized eq kernel for IN list with
column references (#20528)
add 73fbd48070 Upgrade DataFusion to arrow-rs/parquet 58.0.0 /
`object_store` 0.13.0 (#19728)
add d0b3216d21 chore(deps): bump getrandom from 0.3.4 to 0.4.1
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (af6f730837)
\
N -- N -- N refs/heads/dependabot/cargo/main/getrandom-0.4.1
(d0b3216d21)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
Cargo.lock | 167 ++---
Cargo.toml | 18 +-
datafusion-cli/Cargo.toml | 3 +
datafusion-cli/src/exec.rs | 10 +-
datafusion-cli/src/functions.rs | 8 +-
datafusion-cli/src/highlighter.rs | 3 +-
datafusion-cli/src/main.rs | 12 +-
datafusion-cli/src/object_storage.rs | 17 +-
datafusion-cli/src/object_storage/instrumented.rs | 128 ++--
datafusion-cli/src/print_format.rs | 20 +-
datafusion-cli/src/print_options.rs | 16 +-
.../custom_data_source/adapter_serialization.rs | 2 +-
.../examples/custom_data_source/csv_json_opener.rs | 2 +-
.../custom_data_source/custom_datasource.rs | 6 +-
.../custom_data_source/custom_file_casts.rs | 6 +-
.../custom_data_source/default_column_values.rs | 4 +-
.../examples/data_io/json_shredding.rs | 4 +-
.../examples/data_io/parquet_advanced_index.rs | 8 +-
.../memory_pool_execution_plan.rs | 4 +-
.../ffi/ffi_example_table_provider/Cargo.toml | 3 +
.../examples/ffi/ffi_module_interface/Cargo.toml | 3 +
.../examples/ffi/ffi_module_loader/Cargo.toml | 3 +
.../examples/proto/composed_extension_codec.rs | 4 +-
.../examples/relation_planner/table_sample.rs | 6 +-
datafusion/catalog/src/memory/table.rs | 6 +-
datafusion/common/Cargo.toml | 4 +
datafusion/common/benches/scalar_to_array.rs | 107 +++
datafusion/common/src/alias.rs | 10 +
datafusion/common/src/config.rs | 35 +-
datafusion/common/src/file_options/mod.rs | 2 +-
.../common/src/file_options/parquet_writer.rs | 8 +-
datafusion/common/src/scalar/mod.rs | 96 ++-
datafusion/common/src/stats.rs | 2 +-
datafusion/common/src/types/native.rs | 4 +-
datafusion/core/benches/parquet_query_sql.rs | 4 +-
datafusion/core/benches/parquet_struct_query.rs | 4 +-
datafusion/core/benches/push_down_filter.rs | 2 +-
datafusion/core/benches/reset_plan_states.rs | 2 +
datafusion/core/benches/spm.rs | 5 +-
datafusion/core/benches/sql_query_with_io.rs | 2 +-
datafusion/core/src/datasource/file_format/csv.rs | 29 +-
.../core/src/datasource/file_format/parquet.rs | 38 +-
.../core/src/datasource/physical_plan/csv.rs | 2 +-
datafusion/core/src/physical_planner.rs | 26 +-
datafusion/core/src/test/object_store.rs | 65 +-
.../tests/catalog_listing/pruned_partition_list.rs | 2 +-
datafusion/core/tests/custom_sources_cases/mod.rs | 9 +-
.../provider_filter_pushdown.rs | 9 +-
.../core/tests/custom_sources_cases/statistics.rs | 6 +-
datafusion/core/tests/dataframe/mod.rs | 9 +-
.../core/tests/datasource/object_store_access.rs | 113 +---
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/once_exec.rs | 6 +-
datafusion/core/tests/fuzz_cases/pruning.rs | 4 +-
.../spilling_fuzz_in_memory_constrained_env.rs | 16 +-
.../core/tests/fuzz_cases/topk_filter_pushdown.rs | 2 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 2 +-
datafusion/core/tests/parquet/custom_reader.rs | 2 +-
datafusion/core/tests/parquet/expr_adapter.rs | 2 +-
.../core/tests/parquet/external_access_plan.rs | 2 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 4 +-
datafusion/core/tests/parquet/mod.rs | 2 +-
.../physical_optimizer/enforce_distribution.rs | 6 +-
.../tests/physical_optimizer/join_selection.rs | 12 +-
.../tests/physical_optimizer/pushdown_utils.rs | 2 +-
.../replace_with_order_preserving_variants.rs | 2 +-
.../core/tests/physical_optimizer/test_utils.rs | 8 +-
datafusion/core/tests/sql/path_partition.rs | 65 +-
.../core/tests/tracing/traceable_object_store.rs | 34 +-
.../core/tests/user_defined/insert_operation.rs | 20 +-
.../core/tests/user_defined/user_defined_plan.rs | 10 +-
datafusion/datasource-arrow/src/file_format.rs | 3 +-
datafusion/datasource-arrow/src/source.rs | 2 +-
datafusion/datasource-avro/src/file_format.rs | 2 +-
datafusion/datasource-avro/src/source.rs | 2 +-
datafusion/datasource-csv/src/file_format.rs | 4 +-
datafusion/datasource-json/src/file_format.rs | 2 +-
datafusion/datasource-json/src/source.rs | 2 +-
.../benches/parquet_nested_filter_pushdown.rs | 12 +-
datafusion/datasource-parquet/src/file_format.rs | 10 +-
datafusion/datasource-parquet/src/opener.rs | 11 +-
.../datasource-parquet/src/row_group_filter.rs | 3 +-
datafusion/datasource/src/mod.rs | 2 +-
datafusion/datasource/src/sink.rs | 6 +-
datafusion/datasource/src/source.rs | 24 +-
datafusion/datasource/src/url.rs | 42 +-
datafusion/execution/src/memory_pool/mod.rs | 7 +-
datafusion/expr-common/src/signature.rs | 2 +-
datafusion/expr-common/src/type_coercion/binary.rs | 2 +-
datafusion/expr/src/logical_plan/plan.rs | 2 +-
datafusion/expr/src/type_coercion/functions.rs | 2 +-
datafusion/ffi/src/config/extension_options.rs | 288 ++++++++
datafusion/ffi/src/config/mod.rs | 169 +++++
datafusion/ffi/src/execution_plan.rs | 20 +-
datafusion/ffi/src/lib.rs | 1 +
datafusion/ffi/src/session/config.rs | 39 +-
datafusion/ffi/src/tests/async_provider.rs | 8 +-
datafusion/ffi/src/tests/config.rs | 51 ++
datafusion/ffi/src/tests/mod.rs | 6 +
datafusion/ffi/tests/ffi_config.rs | 124 ++++
datafusion/functions-aggregate/src/regr.rs | 2 +-
datafusion/functions-nested/Cargo.toml | 5 +
datafusion/functions-nested/benches/array_has.rs | 273 +++++++-
.../functions-nested/benches/array_position.rs | 237 +++++++
datafusion/functions-nested/src/array_has.rs | 252 ++++++-
datafusion/functions-nested/src/cardinality.rs | 13 +-
datafusion/functions-nested/src/position.rs | 163 ++++-
datafusion/functions-nested/src/range.rs | 25 +-
datafusion/functions-nested/src/set_ops.rs | 2 +-
datafusion/functions-nested/src/string.rs | 49 +-
datafusion/functions-window/src/nth_value.rs | 2 +-
datafusion/functions/src/core/getfield.rs | 2 +-
datafusion/functions/src/datetime/date_bin.rs | 291 +++++---
datafusion/functions/src/math/round.rs | 472 +++++++++++--
datafusion/functions/src/regex/regexpreplace.rs | 48 +-
datafusion/functions/src/string/split_part.rs | 130 +++-
datafusion/optimizer/src/analyzer/type_coercion.rs | 4 -
.../src/decorrelate_predicate_subquery.rs | 2 +-
.../optimizer/src/extract_leaf_expressions.rs | 41 +-
datafusion/optimizer/src/optimizer.rs | 46 +-
datafusion/optimizer/src/push_down_filter.rs | 2 +-
datafusion/physical-expr-adapter/Cargo.toml | 3 +
.../physical-expr-adapter/src/schema_rewriter.rs | 83 ++-
.../src/equivalence/properties/mod.rs | 7 +-
datafusion/physical-expr/src/expressions/case.rs | 2 +-
.../physical-expr/src/expressions/in_list.rs | 245 ++++++-
datafusion/physical-expr/src/partitioning.rs | 2 +-
.../src/simplifier/const_evaluator.rs | 102 ++-
datafusion/physical-expr/src/simplifier/mod.rs | 9 +-
datafusion/physical-expr/src/simplifier/not.rs | 4 +
.../src/combine_partial_final_agg.rs | 2 +-
.../physical-optimizer/src/enforce_distribution.rs | 85 +--
datafusion/physical-optimizer/src/ensure_coop.rs | 6 +-
.../physical-optimizer/src/join_selection.rs | 11 +-
.../physical-optimizer/src/output_requirements.rs | 6 +-
datafusion/physical-plan/src/aggregates/mod.rs | 176 ++++-
.../physical-plan/src/aggregates/row_hash.rs | 171 ++++-
datafusion/physical-plan/src/analyze.rs | 6 +-
datafusion/physical-plan/src/async_func.rs | 27 +-
datafusion/physical-plan/src/buffer.rs | 23 +-
datafusion/physical-plan/src/coalesce_batches.rs | 25 +-
.../physical-plan/src/coalesce_partitions.rs | 26 +-
datafusion/physical-plan/src/coop.rs | 24 +-
datafusion/physical-plan/src/display.rs | 2 +-
datafusion/physical-plan/src/empty.rs | 8 +-
datafusion/physical-plan/src/execution_plan.rs | 74 +-
datafusion/physical-plan/src/explain.rs | 6 +-
datafusion/physical-plan/src/filter.rs | 31 +-
datafusion/physical-plan/src/joins/cross_join.rs | 28 +-
.../physical-plan/src/joins/hash_join/exec.rs | 400 ++++++-----
.../src/joins/hash_join/inlist_builder.rs | 58 +-
.../physical-plan/src/joins/nested_loop_join.rs | 36 +-
.../src/joins/piecewise_merge_join/classic_join.rs | 2 +-
.../src/joins/piecewise_merge_join/exec.rs | 90 ++-
.../src/joins/sort_merge_join/exec.rs | 27 +-
.../src/joins/sort_merge_join/stream.rs | 81 ++-
.../src/joins/sort_merge_join/tests.rs | 453 ++++++++++++-
.../physical-plan/src/joins/symmetric_hash_join.rs | 22 +-
datafusion/physical-plan/src/joins/utils.rs | 2 +-
datafusion/physical-plan/src/limit.rs | 47 +-
datafusion/physical-plan/src/memory.rs | 19 +-
datafusion/physical-plan/src/placeholder_row.rs | 8 +-
datafusion/physical-plan/src/projection.rs | 20 +-
datafusion/physical-plan/src/recursive_query.rs | 6 +-
datafusion/physical-plan/src/repartition/mod.rs | 32 +-
datafusion/physical-plan/src/sorts/mod.rs | 2 +
datafusion/physical-plan/src/sorts/partial_sort.rs | 27 +-
datafusion/physical-plan/src/sorts/sort.rs | 102 ++-
.../src/sorts/sort_preserving_merge.rs | 36 +-
datafusion/physical-plan/src/sorts/stream.rs | 164 ++++-
datafusion/physical-plan/src/spill/mod.rs | 14 +-
.../physical-plan/src/spill/spill_manager.rs | 51 +-
datafusion/physical-plan/src/streaming.rs | 8 +-
datafusion/physical-plan/src/test.rs | 14 +-
datafusion/physical-plan/src/test/exec.rs | 149 ++++-
datafusion/physical-plan/src/topk/mod.rs | 44 +-
datafusion/physical-plan/src/union.rs | 41 +-
datafusion/physical-plan/src/unnest.rs | 24 +-
.../src/windows/bounded_window_agg_exec.rs | 20 +-
datafusion/physical-plan/src/windows/mod.rs | 12 +
.../physical-plan/src/windows/window_agg_exec.rs | 24 +-
datafusion/physical-plan/src/work_table.rs | 8 +-
datafusion/proto/src/physical_plan/to_proto.rs | 3 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 45 ++
datafusion/pruning/src/pruning_predicate.rs | 2 +-
datafusion/spark/src/function/array/shuffle.rs | 16 +-
datafusion/spark/src/function/datetime/last_day.rs | 6 +-
datafusion/spark/src/function/datetime/next_day.rs | 2 +-
.../spark/src/function/map/map_from_arrays.rs | 4 +-
datafusion/spark/src/function/map/utils.rs | 4 +-
datafusion/spark/src/function/math/bin.rs | 109 +++
datafusion/spark/src/function/math/mod.rs | 8 +
datafusion/spark/src/function/math/width_bucket.rs | 4 +-
.../spark/src/function/string/format_string.rs | 2 +-
datafusion/sql/src/expr/function.rs | 2 +-
datafusion/sql/src/planner.rs | 8 +-
datafusion/sqllogictest/README.md | 30 +
datafusion/sqllogictest/bin/sqllogictests.rs | 181 ++++-
datafusion/sqllogictest/test_files/array.slt | 139 +++-
.../sqllogictest/test_files/date_bin_errors.slt | 60 ++
.../test_files/datetime/timestamps.slt | 12 +
datafusion/sqllogictest/test_files/decimal.slt | 2 +-
.../test_files/dynamic_filter_pushdown_config.slt | 2 +-
.../sqllogictest/test_files/explain_tree.slt | 14 +-
datafusion/sqllogictest/test_files/expr.slt | 20 +
datafusion/sqllogictest/test_files/joins.slt | 28 +
datafusion/sqllogictest/test_files/limit.slt | 8 +-
.../test_files/projection_pushdown.slt | 41 ++
.../sqllogictest/test_files/push_down_filter.slt | 745 ---------------------
.../test_files/push_down_filter_outer_joins.slt | 264 ++++++++
.../test_files/push_down_filter_parquet.slt | 188 ++++++
.../test_files/push_down_filter_regression.slt | 200 ++++++
.../test_files/push_down_filter_unnest.slt | 148 ++++
datafusion/sqllogictest/test_files/scalar.slt | 68 +-
.../test_files/spark/array/shuffle.slt | 10 +
.../test_files/spark/datetime/make_interval.slt | 10 +-
.../sqllogictest/test_files/spark/math/bin.slt | 79 ++-
.../sqllogictest/test_files/type_coercion.slt | 21 +
datafusion/sqllogictest/test_files/window.slt | 8 +-
.../logical_plan/consumer/expr/field_reference.rs | 53 +-
.../src/logical_plan/consumer/expr/mod.rs | 5 +-
.../src/logical_plan/consumer/expr/subquery.rs | 44 +-
.../substrait/src/logical_plan/consumer/plan.rs | 5 +-
.../src/logical_plan/consumer/rel/exchange_rel.rs | 3 +-
.../logical_plan/consumer/substrait_consumer.rs | 123 +++-
.../src/logical_plan/producer/expr/cast.rs | 2 +-
.../substrait/tests/cases/consumer_integration.rs | 78 ++-
.../nested_correlated_subquery.substrait.json | 265 ++++++++
datafusion/wasmtest/src/lib.rs | 2 +-
docs/source/conf.py | 1 +
.../library-user-guide/custom-table-providers.md | 6 +-
docs/source/library-user-guide/upgrading/53.0.0.md | 74 ++
docs/source/user-guide/sql/scalar_functions.md | 12 +-
233 files changed, 8040 insertions(+), 2533 deletions(-)
create mode 100644 datafusion/common/benches/scalar_to_array.rs
create mode 100644 datafusion/ffi/src/config/extension_options.rs
create mode 100644 datafusion/ffi/src/config/mod.rs
create mode 100644 datafusion/ffi/src/tests/config.rs
create mode 100644 datafusion/ffi/tests/ffi_config.rs
create mode 100644 datafusion/functions-nested/benches/array_position.rs
create mode 100644 datafusion/spark/src/function/math/bin.rs
create mode 100644 datafusion/sqllogictest/test_files/date_bin_errors.slt
delete mode 100644 datafusion/sqllogictest/test_files/push_down_filter.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_outer_joins.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_parquet.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_regression.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_unnest.slt
create mode 100644
datafusion/substrait/tests/testdata/test_plans/nested_correlated_subquery.substrait.json
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
