This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/proto-33bc8ff79f
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit 759cd3d47e chore(deps): bump pbjson-types from 0.8.0 to 0.9.0 in the
proto group
add c875ef9d24 repro and disable dyn filter for preserve file partitions
(#20175)
add eca6bfe133 chore(deps): bump taiki-e/install-action from 2.67.18 to
2.67.27 (#20254)
add ff15500676 chore(deps): bump setuptools from 80.10.2 to 82.0.0 in
/docs (#20255)
add 7eae96bf1d chore(deps): bump sysinfo from 0.38.0 to 0.38.1 (#20261)
add cbe878ccdd chore(deps): bump clap from 4.5.56 to 4.5.57 (#20265)
add 35caa19b6d chore(deps): bump tempfile from 3.24.0 to 3.25.0 (#20262)
add a41e543326 chore(deps): bump regex from 1.12.2 to 1.12.3 (#20260)
add 86e39afcc1 chore(deps): bump criterion from 0.8.1 to 0.8.2 (#20258)
add 9e86de2865 chore(deps): bump regex-syntax from 0.8.8 to 0.8.9 (#20264)
add b790ed313e chore(deps): bump aws-config from 1.8.12 to 1.8.13 (#20263)
add dff1cad385 chore(deps): bump async-compression from 0.4.37 to 0.4.39
(#20259)
add 69d0f44cc4 Support JSON arrays reader/parse for datafusion (#19924)
add 49776a65e4 chore: Add confirmation before tarball is released (#20207)
add 0f74dbfda1 FilterExec should remap indices of parent dynamic filters
(#20286)
add f9d37cf769 Clean up expression placement UDF usage in tests (#20272)
add 6b6f64d7fb chore(deps): bump the arrow-parquet group with 7 updates
(#20256)
add 17416bf92f Cleanup example metadata parsing utilities(#20251) (#20252)
add 4e2c0f1fea perf: improve performance of
`array_union`/`array_intersect` with batched row conversion (#20243)
add ecf3b502cf Add `StructArray` and `RunArray` benchmark tests to
`with_hashes` (#20182)
add a544b8fc08 fix: disable dynamic filter pushdown for non min/max
aggregates (#20279)
add aa9520e6e2 feat: Add `partition_stats()` for `EmptyExec` (#20203)
add 5fccac12cc Add protoc support for ArrowScanExecNode (#20280) (#20284)
add 85cdf53fab perf: various optimizations to eliminate branch
misprediction in hash_utils (#20168)
add 0401a4732f fix: Avoid integer overflow in substr() (#20199)
add 73bce15f29 fix: Fix scalar broadcast for to_timestamp() (#20224)
add 4d06c40d8d Improve ExternalSorter ResourcesExhausted Error Message
(#20226)
add 9fd84e796a Introduce ProjectionExprs::unproject_exprs/project_exprs
and improve docs (#20193)
add e4e8598be4 fix: Add integer check for bitwise coercion (#20241)
add 2cca3fad33 chore: Remove "extern crate criterion" in benches (#20299)
add af5f470465 Support pushing down empty projections into joins (#20191)
add 2e202fd6fe docs: fix typos and improve wording in README (#20301)
add a3d46510f1 chore: change width_bucket buckets parameter from i32 to
i64 (#20330)
add 51e5c98cf5 fix null handling for `nanvl` & implement fast path (#20205)
add f48dc72220 Remove the statistics() api in execution plan (#20319)
add 9c6a35ffea feat: add ExtractLeafExpressions optimizer rule for
get_field pushdown (#20117)
add 682da846b5 feat: Push limit into hash join (#20228)
add 4ad5c3dab7 perf: Optimize strpos() for ASCII-only inputs (#20295)
add f5a2ac3ec9 fix: percentile_cont interpolation causes NaN for f16 input
(#20208)
add e9bcfb449c perf: Optimize compare_element_to_list (#20323)
add 80485d7516 unify the prettier version adv(#20024) (#20311)
add bdad206a3d chore: Make memchr a workspace dependency (#20345)
add 7d217b1ec3 Reduce ExtractLeafExpressions optimizer overhead with fast
pre-scan (#20341)
add f471aaf275 feat(datafusion-cli): enhance CLI helper with default hint
(#20310)
add 3f38609277 perf: Optimize replace() fastpath by avoiding alloc (#20344)
add 98cc75361f Adds support for ANSI mode in negative function (#20189)
add 53b0ffb93d fix: validate inter-file ordering in eq_properties()
(#20329)
add d47bd599ee Support parent dynamic filters for more join types (#20192)
add b0566c57cc fix: update filter predicates for min/max aggregates only
if bounds change (#20380)
add 132b04369d Fix incorrect `SortExec` removal before `AggregateExec`
(option 2) (#20247)
add 331e1f8fdb chore(deps): bump pbjson-types from 0.8.0 to 0.9.0 in the
proto group
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (759cd3d47e)
\
N -- N -- N refs/heads/dependabot/cargo/main/proto-33bc8ff79f
(331e1f8fdb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/audit.yml | 2 +-
.github/workflows/rust.yml | 4 +-
Cargo.lock | 192 +-
Cargo.toml | 9 +-
README.md | 16 +-
benchmarks/Cargo.toml | 2 +-
ci/scripts/check_examples_docs.sh | 5 +-
ci/scripts/doc_prettier_check.sh | 10 +-
.../{rust_docs.sh => utils/tool_versions.sh} | 8 +-
datafusion-cli/Cargo.toml | 4 +-
datafusion-cli/src/helper.rs | 14 +-
datafusion-cli/src/highlighter.rs | 10 +-
.../cli_top_memory_consumers@no_track.snap | 2 +-
.../snapshots/[email protected] | 2 +-
.../cli_top_memory_consumers@top3_default.snap | 2 +-
.../examples/custom_data_source/csv_json_opener.rs | 1 +
.../memory_pool_execution_plan.rs | 6 +-
.../execution_monitoring/memory_pool_tracking.rs | 3 +-
datafusion-examples/src/utils/csv_to_parquet.rs | 3 +-
datafusion-examples/src/utils/datasets/mod.rs | 8 +-
.../src/utils/example_metadata/discover.rs | 32 +-
.../src/utils/example_metadata/model.rs | 11 +-
.../src/utils/example_metadata/parser.rs | 29 +-
.../src/utils/example_metadata/render.rs | 9 -
datafusion/common/benches/with_hashes.rs | 174 +-
datafusion/common/src/config.rs | 22 +
datafusion/common/src/hash_utils.rs | 208 +-
datafusion/core/Cargo.toml | 2 +-
datafusion/core/benches/aggregate_query_sql.rs | 7 +-
datafusion/core/benches/csv_load.rs | 7 +-
datafusion/core/benches/dataframe.rs | 7 +-
datafusion/core/benches/distinct_query_sql.rs | 8 +-
datafusion/core/benches/math_query_sql.rs | 7 +-
datafusion/core/benches/physical_plan.rs | 6 +-
.../core/benches/range_and_generate_series.rs | 6 +-
datafusion/core/benches/sort_limit_query_sql.rs | 7 +-
datafusion/core/benches/sql_planner.rs | 7 +-
datafusion/core/benches/window_query_sql.rs | 7 +-
datafusion/core/src/dataframe/mod.rs | 2 +-
datafusion/core/src/datasource/file_format/json.rs | 259 ++-
.../core/src/datasource/file_format/options.rs | 61 +-
datafusion/core/src/datasource/listing/table.rs | 3 +-
.../core/src/datasource/physical_plan/json.rs | 12 +-
datafusion/core/src/execution/context/json.rs | 10 +-
datafusion/core/src/prelude.rs | 2 +-
datafusion/core/tests/custom_sources_cases/mod.rs | 4 -
.../provider_filter_pushdown.rs | 8 +-
.../core/tests/custom_sources_cases/statistics.rs | 4 -
datafusion/core/tests/data/json_array.json | 5 +
datafusion/core/tests/data/json_empty_array.json | 1 +
datafusion/core/tests/dataframe/mod.rs | 9 +-
.../physical_optimizer/enforce_distribution.rs | 7 +-
.../tests/physical_optimizer/filter_pushdown.rs | 240 ++-
.../tests/physical_optimizer/join_selection.rs | 4 -
.../tests/physical_optimizer/limit_pushdown.rs | 167 +-
.../physical_optimizer/partition_statistics.rs | 62 +-
.../physical_optimizer/projection_pushdown.rs | 44 +
.../core/tests/user_defined/user_defined_plan.rs | 8 +-
datafusion/datasource-json/Cargo.toml | 2 +
datafusion/datasource-json/src/file_format.rs | 150 +-
datafusion/datasource-json/src/mod.rs | 1 +
datafusion/datasource-json/src/source.rs | 554 +++++-
datafusion/datasource-json/src/utils.rs | 778 ++++++++
datafusion/datasource/Cargo.toml | 2 +-
datafusion/datasource/src/file_scan_config.rs | 194 +-
datafusion/datasource/src/source.rs | 10 -
datafusion/datasource/src/statistics.rs | 3 +-
datafusion/expr-common/src/type_coercion/binary.rs | 4 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 47 +
datafusion/ffi/src/execution_plan.rs | 4 -
.../benches/accumulate.rs | 2 -
datafusion/functions-aggregate/Cargo.toml | 1 +
.../functions-aggregate/src/percentile_cont.rs | 93 +-
datafusion/functions-nested/Cargo.toml | 4 +
.../functions-nested/benches/array_expression.rs | 6 +-
datafusion/functions-nested/benches/array_has.rs | 7 +-
.../functions-nested/benches/array_remove.rs | 7 +-
.../functions-nested/benches/array_repeat.rs | 7 +-
.../functions-nested/benches/array_reverse.rs | 6 +-
.../functions-nested/benches/array_set_ops.rs | 168 ++
datafusion/functions-nested/benches/array_slice.rs | 2 -
datafusion/functions-nested/benches/map.rs | 2 -
datafusion/functions-nested/src/position.rs | 1 -
datafusion/functions-nested/src/set_ops.rs | 150 +-
datafusion/functions-nested/src/utils.rs | 5 +-
datafusion/functions/Cargo.toml | 6 +
datafusion/functions/benches/ascii.rs | 1 -
datafusion/functions/benches/character_length.rs | 2 -
datafusion/functions/benches/chr.rs | 2 -
datafusion/functions/benches/contains.rs | 2 -
datafusion/functions/benches/cot.rs | 2 -
datafusion/functions/benches/crypto.rs | 2 -
datafusion/functions/benches/date_bin.rs | 2 -
datafusion/functions/benches/date_trunc.rs | 2 -
datafusion/functions/benches/encoding.rs | 2 -
datafusion/functions/benches/ends_with.rs | 2 -
datafusion/functions/benches/factorial.rs | 2 -
datafusion/functions/benches/find_in_set.rs | 2 -
datafusion/functions/benches/floor_ceil.rs | 2 -
datafusion/functions/benches/gcd.rs | 2 -
datafusion/functions/benches/initcap.rs | 2 -
datafusion/functions/benches/isnan.rs | 2 -
datafusion/functions/benches/iszero.rs | 2 -
datafusion/functions/benches/left_right.rs | 2 -
datafusion/functions/benches/levenshtein.rs | 2 -
datafusion/functions/benches/lower.rs | 2 -
datafusion/functions/benches/make_date.rs | 2 -
datafusion/functions/benches/nanvl.rs | 114 ++
datafusion/functions/benches/nullif.rs | 2 -
datafusion/functions/benches/pad.rs | 2 -
datafusion/functions/benches/random.rs | 2 -
datafusion/functions/benches/regexp_count.rs | 2 -
datafusion/functions/benches/regx.rs | 2 -
datafusion/functions/benches/repeat.rs | 2 -
datafusion/functions/benches/replace.rs | 2 -
datafusion/functions/benches/reverse.rs | 1 -
datafusion/functions/benches/round.rs | 2 -
datafusion/functions/benches/signum.rs | 2 -
datafusion/functions/benches/split_part.rs | 2 -
datafusion/functions/benches/starts_with.rs | 2 -
datafusion/functions/benches/strpos.rs | 11 +-
datafusion/functions/benches/substr.rs | 2 -
datafusion/functions/benches/substr_index.rs | 2 -
datafusion/functions/benches/to_char.rs | 2 -
datafusion/functions/benches/to_hex.rs | 2 -
datafusion/functions/benches/to_timestamp.rs | 2 -
datafusion/functions/benches/translate.rs | 2 -
datafusion/functions/benches/trim.rs | 2 -
datafusion/functions/benches/trunc.rs | 2 -
datafusion/functions/benches/upper.rs | 2 -
datafusion/functions/benches/uuid.rs | 2 -
datafusion/functions/src/datetime/to_timestamp.rs | 71 +-
datafusion/functions/src/math/nanvl.rs | 96 +-
datafusion/functions/src/string/replace.rs | 20 +-
datafusion/functions/src/unicode/strpos.rs | 68 +-
datafusion/functions/src/unicode/substr.rs | 29 +-
datafusion/optimizer/Cargo.toml | 2 +-
.../optimizer/src/extract_leaf_expressions.rs | 1995 +++++++++++++++++---
.../optimizer/src/optimize_projections/mod.rs | 14 +-
datafusion/optimizer/src/optimizer.rs | 3 +
datafusion/optimizer/src/test/udfs.rs | 20 +-
datafusion/physical-expr/src/projection.rs | 115 +-
.../src/enforce_sorting/sort_pushdown.rs | 74 +
.../physical-optimizer/src/output_requirements.rs | 4 -
datafusion/physical-plan/src/aggregates/mod.rs | 10 +-
.../physical-plan/src/aggregates/no_grouping.rs | 21 +-
datafusion/physical-plan/src/coalesce_batches.rs | 4 -
.../physical-plan/src/coalesce_partitions.rs | 4 -
datafusion/physical-plan/src/display.rs | 4 -
datafusion/physical-plan/src/empty.rs | 36 +-
datafusion/physical-plan/src/execution_plan.rs | 19 -
datafusion/physical-plan/src/filter.rs | 72 +-
datafusion/physical-plan/src/filter_pushdown.rs | 174 +-
datafusion/physical-plan/src/joins/cross_join.rs | 4 -
.../physical-plan/src/joins/hash_join/exec.rs | 313 ++-
.../physical-plan/src/joins/hash_join/stream.rs | 43 +-
.../physical-plan/src/joins/nested_loop_join.rs | 4 -
.../src/joins/sort_merge_join/exec.rs | 4 -
.../physical-plan/src/joins/symmetric_hash_join.rs | 7 +-
datafusion/physical-plan/src/joins/utils.rs | 55 +-
datafusion/physical-plan/src/limit.rs | 8 -
datafusion/physical-plan/src/memory.rs | 6 +-
datafusion/physical-plan/src/placeholder_row.rs | 4 -
datafusion/physical-plan/src/projection.rs | 47 +-
datafusion/physical-plan/src/recursive_query.rs | 6 +-
datafusion/physical-plan/src/repartition/mod.rs | 4 -
datafusion/physical-plan/src/sorts/partial_sort.rs | 4 -
datafusion/physical-plan/src/sorts/sort.rs | 24 +-
.../src/sorts/sort_preserving_merge.rs | 4 -
datafusion/physical-plan/src/test.rs | 4 -
datafusion/physical-plan/src/test/exec.rs | 12 -
datafusion/physical-plan/src/union.rs | 8 -
.../src/windows/bounded_window_agg_exec.rs | 4 -
.../physical-plan/src/windows/window_agg_exec.rs | 4 -
datafusion/physical-plan/src/work_table.rs | 4 -
.../proto-common/proto/datafusion_common.proto | 1 +
datafusion/proto-common/src/from_proto/mod.rs | 1 +
datafusion/proto-common/src/generated/pbjson.rs | 18 +
datafusion/proto-common/src/generated/prost.rs | 3 +
datafusion/proto-common/src/to_proto/mod.rs | 1 +
datafusion/proto/proto/datafusion.proto | 5 +
.../proto/src/generated/datafusion_proto_common.rs | 3 +
datafusion/proto/src/generated/pbjson.rs | 106 ++
datafusion/proto/src/generated/prost.rs | 9 +-
datafusion/proto/src/logical_plan/file_formats.rs | 2 +
datafusion/proto/src/physical_plan/mod.rs | 42 +
.../proto/tests/cases/roundtrip_logical_plan.rs | 4 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 28 +-
datafusion/spark/benches/char.rs | 2 -
datafusion/spark/benches/hex.rs | 2 -
datafusion/spark/benches/sha2.rs | 2 -
datafusion/spark/benches/slice.rs | 2 -
datafusion/spark/benches/space.rs | 2 -
datafusion/spark/benches/substring.rs | 2 -
datafusion/spark/benches/unhex.rs | 2 -
datafusion/spark/src/function/math/negative.rs | 430 +++--
datafusion/spark/src/function/math/width_bucket.rs | 100 +-
datafusion/sqllogictest/Cargo.toml | 2 +-
datafusion/sqllogictest/test_files/aggregate.slt | 2 +-
datafusion/sqllogictest/test_files/array.slt | 54 +-
.../test_files/datetime/arith_date_time.slt | 1 -
.../datetime/arith_timestamp_duration.slt | 2 +-
.../test_files/datetime/timestamps.slt | 30 +
.../test_files/dynamic_filter_pushdown_config.slt | 301 +++
datafusion/sqllogictest/test_files/explain.slt | 8 +
datafusion/sqllogictest/test_files/expr.slt | 2 +-
.../sqllogictest/test_files/information_schema.slt | 2 +
datafusion/sqllogictest/test_files/insert.slt | 2 +-
.../test_files/join_disable_repartition_joins.slt | 4 +-
.../test_files/join_limit_pushdown.slt | 269 +++
datafusion/sqllogictest/test_files/joins.slt | 112 +-
datafusion/sqllogictest/test_files/json.slt | 28 +
datafusion/sqllogictest/test_files/limit.slt | 2 +-
.../test_files/limit_single_row_batches.slt | 2 +-
.../test_files/parquet_sorted_statistics.slt | 2 +-
.../test_files/preserve_file_partitioning.slt | 128 ++
datafusion/sqllogictest/test_files/projection.slt | 13 +-
.../test_files/projection_pushdown.slt | 539 ++++--
.../sqllogictest/test_files/push_down_filter.slt | 9 +-
datafusion/sqllogictest/test_files/scalar.slt | 12 +-
.../sqllogictest/test_files/sort_pushdown.slt | 748 ++++++++
.../test_files/spark/collection/size.slt | 1 -
.../test_files/spark/datetime/time_trunc.slt | 1 -
.../test_files/spark/datetime/trunc.slt | 1 -
.../test_files/spark/math/negative.slt | 52 +
datafusion/sqllogictest/test_files/truncate.slt | 2 +-
datafusion/sqllogictest/test_files/unnest.slt | 2 +-
dev/release/release-tarball.sh | 7 +
dev/update_config_docs.sh | 10 +-
dev/update_function_docs.sh | 9 +-
docs/requirements.txt | 2 +-
docs/source/user-guide/configs.md | 1 +
232 files changed, 8780 insertions(+), 1973 deletions(-)
copy ci/scripts/{rust_docs.sh => utils/tool_versions.sh} (80%)
mode change 100755 => 100644
create mode 100644 datafusion/core/tests/data/json_array.json
create mode 100644 datafusion/core/tests/data/json_empty_array.json
create mode 100644 datafusion/datasource-json/src/utils.rs
create mode 100644 datafusion/functions-nested/benches/array_set_ops.rs
create mode 100644 datafusion/functions/benches/nanvl.rs
create mode 100644 datafusion/sqllogictest/test_files/join_limit_pushdown.slt
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
