This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/checkout-6.0.0
in repository https://gitbox.apache.org/repos/asf/datafusion-sandbox.git
discard 234cc03b6 chore(deps): bump actions/checkout from 5.0.0 to 6.0.0
add 8f9081483 refactor: update cmp and nested data in binary operator
(#18256)
add c1965b662 Fix: topk_aggregate benchmark failing (#18502)
add c728d5481 refactor: Add `assert_or_internal_err!` macro for more
ergonomic internal invariant checks (#18511)
add 8e3f157ce chore: enforce clippy lint needless_pass_by_value to
datafusion-physical-optimizer (#18555)
add 2ae3818f3 chore: enforce clippy lint needless_pass_by_value for
datafusion-sql (#18554)
add f10fcbecc chore: enforce clippy lint needless_pass_by_value to
physical-expr-common (#18556)
add a216d4aee chore: Enforce lint rule `clippy::needless_pass_by_value` to
`datafusion-physical-expr` (#18557)
add 6ab4d216b Force `FileSource` to be constructed with a `Schema` (#18386)
add 49782435f Fix out-of-bounds access in SLT runner (#18562)
add a6fab6bbe feat: support complex expr for prepared statement argument
(#18383)
add 0e6be3045 feat: Implement
`SessionState::create_logical_expr_from_sql_expr` (#18423)
add 1d8bc9b0e [main] Update version to 51.0.0, add Changelog (#18551)
(#18565)
add 812bb357c Make array_reverse faster for List and FixedSizeList (#18500)
add b131cac1b Consolidate custom data source examples (#18142) (#18553)
add 28755b1d7 Normalize partitioned and flat object listing (#18146)
add f162fd325 refactor: include metric output_batches into BaselineMetrics
(#18491)
add 1586cabeb feat: added clippy::needless_pass_by_value lint rule to
datafusion/expr (#18532)
add d8845a616 feat: support nested key for get_field (#18394)
add 900ee658d Support Arrow IPC Stream Files (#18457)
add 1988ffc61 chore(deps): bump taiki-e/install-action from 2.62.47 to
2.62.49 (#18581)
add dc000b89e chore(deps): bump maturin from 1.9.6 to 1.10.0 in /docs
(#18590)
add e4066898b chore: Remove unused `tokio` dependency and clippy (#18598)
add 3fcbd023a minor: enforce `clippy::needless_pass_by_value` for crates
that don't require code changes. (#18586)
add 552dbe4f1 fix: Eliminate consecutive repartitions (#18521)
add 9029ff183 refactor: merge CoalesceAsyncExecInput into CoalesceBatches
(#18540)
add 0c0b5fe7d Update release download links on download page (#18550)
add 6d9ab4539 Enhance the help message for invalid command in
datafusion-cli (#18603)
add d57e215f9 Update Release README.md with latest process (#18549)
add cbf3f50fb Add timezone to date_trunc fast path (#18596)
add 65bd13daa Coalesce batches inside FilterExec (#18604)
add a3a5ca4fb Fix misleading boolean 'null' interval tests (#18620)
add 206656813 Clarify tests for `Interval::and`, `Interval::not`, and add
`Interval::or` tests (#18621)
add a8f0d59de bugfix: correct regression on TableType for into_view
(#18617)
add 03d197464 Separating Benchmarks for physical sorted union over large
columns in SQL planner based on Datatype (#18599)
add cb1f01966 Add RunEndEncoded type coercion (#18561)
add fb765c234 minor: refactor with `assert_or_internal_err!()` in
`datafusion/sql` (#18614)
add a94ec8fc1 docs: fix rustup cmd for adding rust-analyzer (#18605)
add 0d52a1eeb chore: ASF tracking process on `.asf.yaml` (#18636)
add becc71be0 feat: Add `ansi` enable parameter for execution config
(#18635)
add 76b4156af feat: Add evaluate_to_arrays function (#18446)
add aa74ed086 Refactor bit aggregate functions signature (#18593)
add 92a3a3302 fix: `with_param_values` on `LogicalPlan::EmptyRelation`
returns incorrect schema (#18286)
add 62f5cd6ac feat: support named variables & defaults for `CREATE
FUNCTION` (#18450)
add 65f369dcf chore(deps): bump taiki-e/install-action from 2.62.49 to
2.62.50 (#18645)
add 30eaff315 bugfix: select_columns should validate column names (#18623)
add 8226ebf2e Consolidate data io examples (#18142) (#18591)
add 6af5c954b feat: Add new() constructor for CachedParquetFileReader
(#18575)
add e661b33ee Correct implementations of `NullableInterval::and` and
`NullableInterval::or`. (#18625)
add cf0357478 chore: ASF tracking process on `.asf.yaml` (#18652)
add d8c63eef7 feat: support decimal for math functions: power (#18032)
add 2a6f3aafc perf: Improve NLJ for very small right side case (#17562)
add dae99acc3 feat: selectivity metrics (for Explain Analyze) in Hash Join
(#18488)
add 35b536377 Refactor Spark bitshift signature (#18649)
add 8ef884d3c Change default of
`AggregateUDFImpl::supports_null_handling_clause` to `false` (#18441)
add ae9e1dc0f chore(deps): bump crate-ci/typos from 1.39.0 to 1.39.1
(#18667)
add 5053c71af Update docs for aggregate repartition test (#18650)
add 3e2972ae1 chore: Enforce lint rule `clippy::needless_pass_by_value` to
`datafusion-catalog` (#18638)
add e29009fbc [main] Update Changelog (#18592) (#18616)
add e42a0b626 Refactor distinct aggregate implementations to use common
buffer (#18348)
add 377c0fce4 chore: enforce lint rule `clippy::needless_pass_by_value` to
`datafusion-datasource-avro` (#18641)
add 37dbf9ebe Refactor Spark expm1 signature (#18655)
add b477816c8 Enforce explicit opt-in for `WITHIN GROUP` syntax in
aggregate UDAFs (#18607)
add a9ad712d1 chore(core): Enforce lint rule
`clippy::needless_pass_by_value` to `datafusion-core` (#18640)
add 6685bbe50 Refactor substr signature (#18653)
add d88a3b20f minor: Use allow->expect to explicitly suppress Clippy lint
checks (#18686)
add 3bc82adbe chore(deps): bump taiki-e/install-action from 2.62.50 to
2.62.51 (#18693)
add eefb680fe chore(deps): bump crate-ci/typos from 1.39.1 to 1.39.2
(#18694)
add f17cc09fb Remove FilterExec from CoalesceBatches optimization rule,
add fetch support (#18630)
add 0850cecc6 minor: refactor with `assert_or_internal_err!()` in
`datafusion/datasource` (#18697)
add 1dddf034f chore: Enforce lint rule `clippy::needless_pass_by_value` to
datafusion-datasource (#18682)
add 2b3b22083 feat: Handle edge case with `corr` with single row and `NaN`
(#18677)
add cfb26b6c0 [main] Update changelog for 51.0.0 RC2 (#18710)
add c39334f65 Refactor Spark crc32/sha1 signatures (#18662)
add 729bf2631 feat: support spark csc (#18642)
add 302dbc8e4 CI: try free up space in `Rust / cargo test (amd64)` action
(#18709)
add e730101c4 chore: enforce clippy lint needless_pass_by_value to
datafusion-proto (#18715)
add 0cfc1fefa chore: enforce clippy lint needless_pass_by_value to
datafusion-spark (#18714)
add 17e6c8815 minor: refactor with `assert_or_internal_err!()` in
`datafusion/optimizer` (#18699)
add a113a83ef minor: refactor with `assert_or_internal_err!()` in
`datafusion/functions` (#18700)
add 1b1cd0d1e minor: refactor with `assert_or_internal_err!()` in
`datafusion/expr-common` (#18702)
add 31b297578 minor: refactor with `assert_or_internal_err!()` in
`datafusion/functions-aggregate` (#18716)
add 899653f69 chore: enforce clippy lint needless_pass_by_value to
datafusion-execution (#18723)
add 30b0982f2 minor: refactor with `assert_or_internal_err!()` in
`datafusion/functions-nested` (#18724)
add 0cd6a19bc chore: enforce clippy lint needless_pass_by_value to
datafusion-substrait (#18703)
add 17050a9c6 chore: Refactor with assert_or_internal_err!() in
datafusion/spark. (#18674)
add 3ac3d0edf Minor: Add docs to release/README.md about rate limits
(#18704)
add 91eba1dfc feat: support spark sec (#18728)
add 3863267af Consolidate query planning examples (#18142) (#18690)
add a26afcea7 minor: refactor with `assert_or_internal_err!()` in
`datafusion/physical-expr-common` (#18735)
add 63545ae59 minor: refactor with `assert_or_internal_err!()` in
`datafusion/physical-expr` (#18736)
add 97af468f3 Consolidate ArrowFileSource and ArrowStreamFileSource
(#18720)
add ac9bb17d1 minor: refactor with `assert_or_internal_err!()` in
`datafusion/physical-optimizer` (#18732)
add 6cb3d7ec4 refactor: reduce duplication in make_udf_function macro
(#18733)
add b0015d3a9 feat(parquet): Implement `scan_efficiency_ratio` metric for
parquet reading (#18577)
add 85777c4f7 minor: refactor with `assert_or_internal_err!()` in
`datafusion/physical-plan` (#18730)
add 79f2b14dc chore: enforce clippy lint needless_pass_by_value to
datafusion-functions-aggregate-common (#18741)
add fad5f8a62 Optimize NullState::build (#18737)
add 8a75075a0 [Minor] Remove RawTableAllocExt (#18748)
add 5791822e4 chore: enforce clippy lint needless_pass_by_value to
datafusion-datasource-parquet (#18695)
add 4198c5a74 minor: refactor with `assert_or_internal_err!()` in
`datafusion/expr` (#18731)
add 0424eaa4d fix: Nested arrays should not get a field in lookup (#18745)
add e5b6d9700 minor: Fix an example in the `PruningPredicate`
documentation (#18742)
add 2437e0303 docs: fix broken catalog example links (#18765)
add e557d0aaa chore(deps): bump indicatif from 0.18.2 to 0.18.3 (#18756)
add b05d81114 doc: Add documentation for error handling (#18762)
add 7fa63789f Consolidate `EliminateNestedUnion` and `EliminateOneUnion`
optimizer rules' (#18678)
add 5d5a276f1 Fix map_query_sql benchmark duplicate key error (#18427)
add 8b8bdc4ad minor: enforce lint rule clippy::needless_pass_by_value to
datafusion-ffi (#18764)
add af2233675 perf: improve performance of `vectorized_equal_to` for
`PrimitiveGroupValueBuilder` in multi group by aggregation (#17977)
add e4bc51481 feat: Enhance map handling to support NULL map values
(#18531)
add bd30fe237 Rename boolean `Interval` constants to match
`NullableInterval` (#18654)
add 71fcd03d6 feat: add RESET statement for configuration variabless
(#18408)
add e64de0f2f chore(deps): bump bytes from 1.10.1 to 1.11.0 (#18755)
add 82181ac48 fix: update schema's data type for `LogicalPlan::Values`
after placeholder substitution (#18740)
add a732dc307 feat: add human-readable formatting to EXPLAIN ANALYZE
metrics #18689 (#18734)
add 0304cda4f CI: Fix `main` branch CI test failure (#18792)
add 958a6de9d chore: Enforce 'clippy::needless_pass_by_value' to
datafusion-expr-common (#18775)
add 19865b327 chore: Finish refactor with `assert_or_internal_err!()`
(#18790)
add 5765a08c8 Switch from xz2 to liblzma to reduce duplicate dependencies
(#17509)
add a07ddb21c chore(deps): bump taiki-e/install-action from 2.62.51 to
2.62.53 (#18796)
add 1b42a7ca7 chore(deps): bump actions/checkout from 5.0.0 to 5.0.1
(#18797)
add b3ff6d83b Misc improvements to ProjectionExprs (#18719)
add 984d21003 fix: Pick correct columns in Sort Merge Equijoin (#18772)
add b2e99fc09 Fix incorrect link for sql_query.rs example in README
(#18807)
add 45e93529d docs: Fix the examples for char_length() and
character_length() (#18808)
add 6601959fe Adds prefix filtering for table URLs (#18780)
add 486c5d860 Refactor InListExpr to support structs by re-using existing
hashing infrastructure (#18449)
add 2dd17b9e6 chore: Add script to protect RC branches during the release
(#18660)
add 373b0ad0f fix: remove `WorkTableExec` special case in
`reset_plan_states` (#18803)
add 794660482 Prevent overflow and panics when casting DATE to TIMESTAMP
by validating bounds (#18761)
add ac9c6b44e feat: support Spark-compatible `abs` math function part 1 -
non-ANSI mode (#18205)
add 307ec5093 chore(deps): bump taiki-e/install-action from 2.62.53 to
2.62.54 (#18815)
add d9d847e22 CI : Enforce clippy: :needless_pass_by_value rule to
datafusion-functions-aggregate (#18805)
add b1af8fe9c Consolidate sql operations examples (#18142) (#18743)
add d01c0d39b chore: Support 'untake' for unassigning github issues
(#18637)
add f9231fc12 Move `GuaranteeRewriter` to datafusion_expr (#18821)
add 5b0aa37c8 Refactor state management in `HashJoinExec` and use CASE
expressions for more precise filters (#18451)
add 3b390d782 Refactor avg & sum signatures away from user defined (#18769)
add 068f96f04 Hash UnionArrays (#18718)
add 5dbbac698 CI: add clippy::needless_pass_by_value rule to
datafusion-functions-window crate (#18838)
add 7fa2a694b Add field to DynamicPhysicalExpr to indicate when the filter
is complete or updated (#18799)
add 0bd127fd9 #17801 Improve nullability reporting of case expressions
(#17813)
add 356616e0c Consolidate execution monitoring examples (#18142) (#18846)
add 99ab87c57 Implement CatalogProviderList in FFI (#18657)
add dc6e7cf86 fix: display the failed sqllogictest file and query that
failed in case of a panic (#18785)
add 1833093ce Removed incorrect union check in enforce_sorting and updated
tests (#18661)
add 7d8b8602a Change `CacheAccessor::remove` to take `&self` rather than
`&mut self` (#18726)
add 8a91db56b Move statistics handling into FileScanConfig (#18721)
add f1ecaccd1 chore: Add filtered pending PRs link to main page (#18854)
add 195bd5cb3 chore(deps): bump actions/checkout from 5.0.1 to 6.0.0
(#18865)
add d65fb860d Remove unnecessary bit counting code from spark `bit_count`
(#18841)
add 78a4bbab6 Fix async_udf batch size behaviour (#18819)
add a7b2e8556 Docs: Enhance contributor guide with testing section (#18852)
add e6ddb48f4 Fix Partial AggregateExec correctness issue dropping rows
(#18712)
add e6d1773ef chore: Add missing boolean tests to `bit_count` Spark
function (#18871)
add c7b339e0b Consolidate proto examples (#18142) (#18861)
add 12cb4cae1 Use logical null count in `case_when_with_expr` (#18872)
add c141fca12 chore: enforce `clippy::needless_pass_by_value` to
`datafusion-physical-plan` (#18864)
add 5e36b052e Refactor spark `bit_get()` signature away from user defined
(#18836)
add 1e9c1f426 minor: enforce lint rule clippy::needless_pass_by_value to
datafusion-functions (#18768)
add 240e2e39a chore: enforce clippy lint needless_pass_by_value to
datafusion-functions-nested (#18839)
add 8b3725aec chore: fix CI on main (#18876)
add c8d26ba01 feat: Support Show runtime settings (#18564)
add 5f2272219 chore: update Repartition DisplayAs to indicate maintained
sort order (#18673)
add 6ed22bb7c implement sum for durations (#18853)
add fbe1ae6ab feat(small): Support `<slt:ignore>` marker in `sqllogictest`
for non-deterministic expected parts (#18857)
add d3ae1a785 Consolidate dataframe examples (#18142) (#18862)
add e50b9398d Docs: Enhance testing documentation with examples and links
(#18851)
add 7d87976e1 Avoid the need to rewrite expressions when evaluating
logical case nullability (#18849)
add fc77be945 chore: remove `pyarrow` feature (#18528)
add b0b43a311 Merge branch 'main' into sandbox-main
add 3fec574ba chore(deps): bump actions/checkout from 5.0.0 to 6.0.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (234cc03b6)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/checkout-6.0.0 (3fec574ba)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/audit.yml | 2 +-
.github/workflows/rust.yml | 48 +-
.github/workflows/take.yml | 32 +-
Cargo.lock | 317 ++--
Cargo.toml | 81 +-
ci/scripts/rust_clippy.sh | 2 +-
datafusion-cli/src/exec.rs | 4 +-
datafusion-examples/Cargo.toml | 5 -
datafusion-examples/README.md | 69 +-
.../examples/builtin_functions/date_time.rs | 2 +
.../examples/builtin_functions/function_factory.rs | 2 +
.../examples/builtin_functions/main.rs | 5 +
.../examples/builtin_functions/regexp.rs | 2 +
.../{ => custom_data_source}/csv_json_opener.rs | 27 +-
.../{ => custom_data_source}/csv_sql_streaming.rs | 5 +-
.../{ => custom_data_source}/custom_datasource.rs | 5 +-
.../{ => custom_data_source}/custom_file_casts.rs | 6 +-
.../{ => custom_data_source}/custom_file_format.rs | 81 +-
.../default_column_values.rs | 48 +-
.../file_stream_provider.rs | 38 +-
.../examples/custom_data_source/main.rs | 135 ++
.../examples/{ => data_io}/catalog.rs | 17 +-
.../examples/{ => data_io}/json_shredding.rs | 5 +-
datafusion-examples/examples/data_io/main.rs | 160 ++
.../parquet_advanced_index.rs} | 20 +-
.../{ => data_io}/parquet_embedded_index.rs | 58 +-
.../examples/{ => data_io}/parquet_encrypted.rs | 6 +-
.../{ => data_io}/parquet_encrypted_with_kms.rs | 5 +-
.../examples/{ => data_io}/parquet_exec_visitor.rs | 15 +-
.../examples/{ => data_io}/parquet_index.rs | 10 +-
.../query_http_csv.rs} | 8 +-
.../examples/{ => data_io}/remote_catalog.rs | 6 +-
.../examples/{ => dataframe}/dataframe.rs | 13 +-
.../{ => dataframe}/deserialize_to_struct.rs | 5 +-
.../{builtin_functions => dataframe}/main.rs | 45 +-
.../main.rs | 57 +-
.../memory_pool_execution_plan.rs | 6 +-
.../memory_pool_tracking.rs | 9 +-
.../examples/{ => execution_monitoring}/tracing.rs | 6 +-
datafusion-examples/examples/flight/client.rs | 2 +
datafusion-examples/examples/flight/main.rs | 5 +
datafusion-examples/examples/flight/server.rs | 2 +
datafusion-examples/examples/flight/sql_server.rs | 2 +
.../{ => proto}/composed_extension_codec.rs | 16 +-
.../examples/{builtin_functions => proto}/main.rs | 39 +-
.../examples/{ => query_planning}/analyzer_rule.rs | 5 +-
.../examples/{ => query_planning}/expr_api.rs | 5 +-
.../examples/query_planning/main.rs | 138 ++
.../{ => query_planning}/optimizer_rule.rs | 5 +-
.../{ => query_planning}/parse_sql_expr.rs | 10 +-
.../examples/{ => query_planning}/plan_to_sql.rs | 20 +-
.../examples/{ => query_planning}/planner_api.rs | 5 +-
.../examples/{ => query_planning}/pruning.rs | 10 +-
.../examples/{ => query_planning}/thread_pools.rs | 7 +-
.../{sql_analysis.rs => sql_ops/analysis.rs} | 273 ++--
.../{sql_dialect.rs => sql_ops/dialect.rs} | 5 +-
.../{sql_frontend.rs => sql_ops/frontend.rs} | 4 +-
.../{builtin_functions => sql_ops}/main.rs | 55 +-
.../examples/{sql_query.rs => sql_ops/query.rs} | 5 +-
datafusion-examples/examples/udf/advanced_udaf.rs | 2 +
datafusion-examples/examples/udf/advanced_udf.rs | 2 +
datafusion-examples/examples/udf/advanced_udwf.rs | 2 +
datafusion-examples/examples/udf/async_udf.rs | 2 +
datafusion-examples/examples/udf/main.rs | 5 +
datafusion-examples/examples/udf/simple_udaf.rs | 2 +
datafusion-examples/examples/udf/simple_udf.rs | 2 +
datafusion-examples/examples/udf/simple_udtf.rs | 2 +
datafusion-examples/examples/udf/simple_udwf.rs | 2 +
datafusion/catalog-listing/Cargo.toml | 1 -
datafusion/catalog-listing/src/helpers.rs | 573 ++-----
datafusion/catalog-listing/src/mod.rs | 3 +
datafusion/catalog-listing/src/table.rs | 38 +-
datafusion/catalog/src/async.rs | 2 +-
datafusion/catalog/src/catalog.rs | 4 +-
datafusion/catalog/src/information_schema.rs | 37 +-
datafusion/catalog/src/lib.rs | 2 +
datafusion/common-runtime/src/lib.rs | 3 +
datafusion/common/Cargo.toml | 4 +-
datafusion/common/src/config.rs | 144 +-
datafusion/common/src/error.rs | 239 ++-
datafusion/common/src/hash_utils.rs | 333 +++-
datafusion/common/src/lib.rs | 2 -
datafusion/common/src/pyarrow.rs | 169 --
datafusion/common/src/scalar/mod.rs | 281 +++-
datafusion/common/src/utils/mod.rs | 9 +-
datafusion/common/src/utils/proxy.rs | 76 +-
datafusion/core/Cargo.toml | 5 +-
datafusion/core/benches/map_query_sql.rs | 10 +-
datafusion/core/benches/sql_planner.rs | 71 +-
datafusion/core/benches/topk_aggregate.rs | 30 +-
datafusion/core/src/dataframe/mod.rs | 22 +-
datafusion/core/src/datasource/file_format/mod.rs | 6 +-
datafusion/core/src/datasource/mod.rs | 13 +-
.../core/src/datasource/physical_plan/avro.rs | 29 +-
.../core/src/datasource/physical_plan/csv.rs | 185 ++-
.../core/src/datasource/physical_plan/json.rs | 24 +-
.../core/src/datasource/physical_plan/parquet.rs | 61 +-
datafusion/core/src/execution/context/mod.rs | 96 +-
datafusion/core/src/execution/session_state.rs | 40 +
datafusion/core/src/lib.rs | 2 +
datafusion/core/src/physical_planner.rs | 101 +-
datafusion/core/src/test/mod.rs | 31 +-
datafusion/core/src/test_util/parquet.rs | 37 +-
.../core/tests/{catalog => catalog_listing}/mod.rs | 2 +-
.../tests/catalog_listing/pruned_partition_list.rs | 251 +++
datafusion/core/tests/core_integration.rs | 3 +
.../part=123/data.arrow | Bin 0 -> 1608 bytes
.../part=456/data.arrow | Bin 0 -> 1608 bytes
datafusion/core/tests/dataframe/mod.rs | 251 ++-
.../core/tests/datasource/object_store_access.rs | 63 +-
datafusion/core/tests/execution/mod.rs | 1 +
datafusion/core/tests/execution/register_arrow.rs | 90 ++
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 697 ++++----
datafusion/core/tests/fuzz_cases/pruning.rs | 5 +-
datafusion/core/tests/macro_hygiene/mod.rs | 1 +
datafusion/core/tests/optimizer/mod.rs | 26 +-
datafusion/core/tests/parquet/custom_reader.rs | 3 +-
.../core/tests/parquet/external_access_plan.rs | 11 +-
datafusion/core/tests/parquet/page_pruning.rs | 4 +-
datafusion/core/tests/parquet/schema_adapter.rs | 22 +-
datafusion/core/tests/parquet/schema_coercion.rs | 15 +-
.../physical_optimizer/enforce_distribution.rs | 523 +++---
.../tests/physical_optimizer/enforce_sorting.rs | 49 +-
.../physical_optimizer/filter_pushdown/mod.rs | 318 +++-
.../physical_optimizer/filter_pushdown/util.rs | 113 +-
.../tests/physical_optimizer/limit_pushdown.rs | 7 +-
.../physical_optimizer/projection_pushdown.rs | 59 +-
.../replace_with_order_preserving_variants.rs | 120 +-
.../tests/physical_optimizer/sanity_checker.rs | 10 +-
.../core/tests/physical_optimizer/test_utils.rs | 14 +-
.../schema_adapter_integration_tests.rs | 42 +-
datafusion/core/tests/sql/explain_analyze.rs | 165 +-
datafusion/core/tests/sql/joins.rs | 30 +-
datafusion/core/tests/sql/mod.rs | 18 +-
datafusion/core/tests/sql/select.rs | 42 +-
datafusion/core/tests/tpcds_planning.rs | 5 +-
datafusion/core/tests/user_defined/mod.rs | 3 +
.../user_defined_async_scalar_functions.rs | 139 ++
.../core/tests/user_defined/user_defined_plan.rs | 19 +-
.../user_defined/user_defined_scalar_functions.rs | 233 ++-
datafusion/datasource-arrow/src/file_format.rs | 436 +++--
datafusion/datasource-arrow/src/mod.rs | 5 +
datafusion/datasource-arrow/src/source.rs | 566 +++++--
.../tests/data/example_stream.arrow | Bin 0 -> 1480 bytes
.../example_stream_corrupted_metadata_length.arrow | Bin 0 -> 1480 bytes
.../tests/data/example_stream_empty.arrow | Bin 0 -> 776 bytes
.../src/avro_to_arrow/arrow_array_reader.rs | 98 +-
.../datasource-avro/src/avro_to_arrow/reader.rs | 8 +-
datafusion/datasource-avro/src/file_format.rs | 10 +-
datafusion/datasource-avro/src/mod.rs | 2 +
datafusion/datasource-avro/src/source.rs | 47 +-
datafusion/datasource-csv/src/file_format.rs | 36 +-
datafusion/datasource-csv/src/mod.rs | 7 +-
datafusion/datasource-csv/src/source.rs | 143 +-
datafusion/datasource-json/src/file_format.rs | 11 +-
datafusion/datasource-json/src/mod.rs | 3 +
datafusion/datasource-json/src/source.rs | 37 +-
datafusion/datasource-parquet/src/access_plan.rs | 17 +-
datafusion/datasource-parquet/src/file_format.rs | 20 +-
datafusion/datasource-parquet/src/metadata.rs | 32 +-
datafusion/datasource-parquet/src/metrics.rs | 15 +-
datafusion/datasource-parquet/src/mod.rs | 3 +
datafusion/datasource-parquet/src/opener.rs | 2 +-
datafusion/datasource-parquet/src/page_filter.rs | 1 +
datafusion/datasource-parquet/src/reader.rs | 56 +-
.../datasource-parquet/src/row_group_filter.rs | 16 +-
datafusion/datasource-parquet/src/source.rs | 103 +-
datafusion/datasource/Cargo.toml | 6 +-
datafusion/datasource/src/file.rs | 13 +-
datafusion/datasource/src/file_compression_type.rs | 4 +-
datafusion/datasource/src/file_format.rs | 5 +-
datafusion/datasource/src/file_scan_config.rs | 221 ++-
datafusion/datasource/src/file_stream.rs | 4 +-
datafusion/datasource/src/memory.rs | 25 +-
datafusion/datasource/src/mod.rs | 4 +
datafusion/datasource/src/sink.rs | 10 +-
datafusion/datasource/src/source.rs | 9 +-
datafusion/datasource/src/statistics.rs | 39 +-
datafusion/datasource/src/table_schema.rs | 6 +
datafusion/datasource/src/test_util.rs | 52 +-
datafusion/datasource/src/url.rs | 85 +-
datafusion/datasource/src/write/demux.rs | 4 +-
datafusion/doc/src/lib.rs | 3 +
datafusion/execution/src/cache/cache_manager.rs | 2 +-
datafusion/execution/src/cache/cache_unit.rs | 12 +-
datafusion/execution/src/cache/mod.rs | 2 +-
datafusion/execution/src/disk_manager.rs | 26 +-
datafusion/execution/src/lib.rs | 3 +
datafusion/execution/src/memory_pool/mod.rs | 107 +-
datafusion/execution/src/memory_pool/pool.rs | 8 +-
datafusion/execution/src/runtime_env.rs | 121 +-
datafusion/expr-common/src/columnar_value.rs | 206 ++-
datafusion/expr-common/src/interval_arithmetic.rs | 1101 ++++++++++---
datafusion/expr-common/src/lib.rs | 3 +
datafusion/expr-common/src/statistics.rs | 160 +-
datafusion/expr-common/src/type_coercion/binary.rs | 84 +-
.../src/type_coercion/binary/tests/mod.rs | 1 +
.../type_coercion/binary/tests/run_end_encoded.rs | 99 ++
datafusion/expr/src/execution_props.rs | 1 +
datafusion/expr/src/expr_fn.rs | 5 +
datafusion/expr/src/expr_rewriter/guarantees.rs | 668 ++++++++
datafusion/expr/src/expr_rewriter/mod.rs | 5 +
datafusion/expr/src/expr_rewriter/order_by.rs | 6 +-
datafusion/expr/src/expr_schema.rs | 220 ++-
datafusion/expr/src/lib.rs | 4 +
datafusion/expr/src/literal.rs | 3 +
datafusion/expr/src/logical_plan/invariants.rs | 20 +-
datafusion/expr/src/logical_plan/mod.rs | 5 +-
datafusion/expr/src/logical_plan/plan.rs | 126 +-
datafusion/expr/src/logical_plan/statement.rs | 12 +
datafusion/expr/src/predicate_bounds.rs | 681 ++++++++
datafusion/expr/src/udaf.rs | 10 +-
datafusion/expr/src/udf.rs | 21 +-
datafusion/expr/src/utils.rs | 7 +-
datafusion/ffi/src/arrow_wrappers.rs | 5 +-
datafusion/ffi/src/catalog_provider.rs | 2 +-
datafusion/ffi/src/catalog_provider_list.rs | 283 ++++
datafusion/ffi/src/lib.rs | 4 +
datafusion/ffi/src/tests/async_provider.rs | 4 +-
datafusion/ffi/src/tests/catalog.rs | 57 +-
datafusion/ffi/src/tests/mod.rs | 6 +
.../tests/{ffi_integration.rs => ffi_catalog.rs} | 74 +-
datafusion/ffi/tests/ffi_integration.rs | 27 -
.../src/aggregate/count_distinct/native.rs | 70 +-
.../src/aggregate/groups_accumulator/accumulate.rs | 10 +-
.../src/aggregate/sum_distinct/numeric.rs | 78 +-
datafusion/functions-aggregate-common/src/lib.rs | 4 +
.../src/noop_accumulator.rs | 70 +
.../functions-aggregate-common/src/tdigest.rs | 1 +
datafusion/functions-aggregate-common/src/utils.rs | 99 +-
.../functions-aggregate/src/approx_distinct.rs | 52 +-
.../src/approx_percentile_cont.rs | 8 +-
.../src/approx_percentile_cont_with_weight.rs | 6 +-
datafusion/functions-aggregate/src/array_agg.rs | 20 +-
datafusion/functions-aggregate/src/average.rs | 55 +-
.../functions-aggregate/src/bit_and_or_xor.rs | 33 +-
datafusion/functions-aggregate/src/correlation.rs | 137 +-
datafusion/functions-aggregate/src/first_last.rs | 116 +-
datafusion/functions-aggregate/src/lib.rs | 3 +
datafusion/functions-aggregate/src/median.rs | 73 +-
datafusion/functions-aggregate/src/nth_value.rs | 22 +-
.../functions-aggregate/src/percentile_cont.rs | 93 +-
datafusion/functions-aggregate/src/sum.rs | 136 +-
.../functions-nested/benches/array_reverse.rs | 76 +-
datafusion/functions-nested/src/array_has.rs | 35 +-
datafusion/functions-nested/src/flatten.rs | 12 +-
datafusion/functions-nested/src/lib.rs | 3 +
datafusion/functions-nested/src/map.rs | 410 ++++-
datafusion/functions-nested/src/position.rs | 20 +-
datafusion/functions-nested/src/remove.rs | 10 +-
datafusion/functions-nested/src/replace.rs | 14 +-
datafusion/functions-nested/src/reverse.rs | 117 +-
datafusion/functions-nested/src/set_ops.rs | 15 +-
datafusion/functions-nested/src/string.rs | 60 +-
datafusion/functions-table/src/lib.rs | 3 +
datafusion/functions-window-common/src/lib.rs | 3 +
datafusion/functions-window/src/lead_lag.rs | 2 +-
datafusion/functions-window/src/lib.rs | 3 +
datafusion/functions-window/src/nth_value.rs | 2 +-
datafusion/functions-window/src/ntile.rs | 4 +-
datafusion/functions-window/src/utils.rs | 4 +-
datafusion/functions/src/core/getfield.rs | 52 +-
datafusion/functions/src/core/greatest.rs | 14 +-
.../functions/src/core/greatest_least_utils.rs | 27 +-
datafusion/functions/src/core/least.rs | 14 +-
datafusion/functions/src/core/mod.rs | 2 +
datafusion/functions/src/crypto/basic.rs | 13 +-
datafusion/functions/src/datetime/common.rs | 8 +-
datafusion/functions/src/datetime/date_trunc.rs | 177 ++-
datafusion/functions/src/datetime/to_char.rs | 14 +-
datafusion/functions/src/lib.rs | 3 +
datafusion/functions/src/macros.rs | 10 +-
datafusion/functions/src/math/abs.rs | 16 +-
datafusion/functions/src/math/monotonicity.rs | 20 +-
datafusion/functions/src/math/pi.rs | 10 +-
datafusion/functions/src/math/power.rs | 448 +++++-
datafusion/functions/src/math/random.rs | 10 +-
datafusion/functions/src/regex/regexpcount.rs | 38 +-
datafusion/functions/src/regex/regexpinstr.rs | 1 +
datafusion/functions/src/regex/regexpreplace.rs | 1 +
datafusion/functions/src/string/ascii.rs | 8 +-
datafusion/functions/src/string/common.rs | 1 +
datafusion/functions/src/string/repeat.rs | 8 +-
datafusion/functions/src/string/split_part.rs | 1 +
datafusion/functions/src/string/uuid.rs | 10 +-
.../functions/src/unicode/character_length.rs | 8 +-
datafusion/functions/src/unicode/find_in_set.rs | 29 +-
datafusion/functions/src/unicode/lpad.rs | 12 +-
datafusion/functions/src/unicode/reverse.rs | 6 +-
datafusion/functions/src/unicode/rpad.rs | 12 +-
datafusion/functions/src/unicode/strpos.rs | 22 +-
datafusion/functions/src/unicode/substr.rs | 106 +-
datafusion/functions/src/utils.rs | 62 +-
.../src/decorrelate_predicate_subquery.rs | 13 +-
datafusion/optimizer/src/eliminate_one_union.rs | 121 --
.../optimizer/src/extract_equijoin_predicate.rs | 15 +-
datafusion/optimizer/src/lib.rs | 9 +-
.../optimizer/src/optimize_projections/mod.rs | 28 +-
...liminate_nested_union.rs => optimize_unions.rs} | 43 +-
datafusion/optimizer/src/optimizer.rs | 7 +-
.../optimizer/src/propagate_empty_relation.rs | 4 +-
datafusion/optimizer/src/push_down_filter.rs | 20 +-
.../optimizer/src/scalar_subquery_to_join.rs | 18 +-
.../src/simplify_expressions/expr_simplifier.rs | 10 +-
.../src/simplify_expressions/guarantees.rs | 476 ------
.../optimizer/src/simplify_expressions/mod.rs | 3 +-
.../optimizer/src/single_distinct_to_groupby.rs | 18 +-
datafusion/physical-expr-adapter/src/lib.rs | 3 +
datafusion/physical-expr-common/src/datum.rs | 83 +-
datafusion/physical-expr-common/src/lib.rs | 3 +
.../physical-expr-common/src/physical_expr.rs | 23 +-
datafusion/physical-expr-common/src/utils.rs | 21 +
datafusion/physical-expr/Cargo.toml | 4 +
datafusion/physical-expr/src/aggregate.rs | 9 +-
datafusion/physical-expr/src/analysis.rs | 43 +-
.../physical-expr/src/async_scalar_function.rs | 25 +-
.../src/equivalence/properties/mod.rs | 2 +-
datafusion/physical-expr/src/expressions/binary.rs | 117 +-
.../src/expressions/binary/kernels.rs | 6 +-
datafusion/physical-expr/src/expressions/case.rs | 404 ++++-
.../src/expressions/dynamic_filters.rs | 102 +-
.../physical-expr/src/expressions/in_list.rs | 1677 +++++++++++++++++---
datafusion/physical-expr/src/expressions/like.rs | 22 +-
.../physical-expr/src/expressions/literal.rs | 1 +
datafusion/physical-expr/src/expressions/not.rs | 8 +-
.../physical-expr/src/intervals/cp_solver.rs | 79 +-
datafusion/physical-expr/src/lib.rs | 3 +
datafusion/physical-expr/src/projection.rs | 194 ++-
.../physical-expr/src/statistics/stats_solver.rs | 2 +-
datafusion/physical-expr/src/utils/guarantee.rs | 4 +-
.../src/window/standard_window_function_expr.rs | 9 +-
datafusion/physical-expr/src/window/window_expr.rs | 9 +-
.../src/coalesce_async_exec_input.rs | 71 -
.../physical-optimizer/src/coalesce_batches.rs | 31 +-
.../physical-optimizer/src/enforce_distribution.rs | 3 +-
.../physical-optimizer/src/enforce_sorting/mod.rs | 7 +-
.../replace_with_order_preserving_variants.rs | 23 +-
.../src/enforce_sorting/sort_pushdown.rs | 4 +-
.../physical-optimizer/src/filter_pushdown.rs | 58 +-
datafusion/physical-optimizer/src/lib.rs | 4 +-
datafusion/physical-optimizer/src/optimizer.rs | 2 -
.../physical-optimizer/src/projection_pushdown.rs | 4 +-
.../group_values/multi_group_by/primitive.rs | 114 +-
.../src/aggregates/group_values/null_builder.rs | 8 +
.../src/aggregates/group_values/row.rs | 14 +-
datafusion/physical-plan/src/aggregates/mod.rs | 71 +-
.../physical-plan/src/aggregates/no_grouping.rs | 23 +-
.../physical-plan/src/aggregates/row_hash.rs | 225 ++-
.../physical-plan/src/aggregates/topk/heap.rs | 14 +-
.../physical-plan/src/aggregates/topk_stream.rs | 11 +-
datafusion/physical-plan/src/analyze.rs | 22 +-
datafusion/physical-plan/src/async_func.rs | 10 +-
datafusion/physical-plan/src/coalesce/mod.rs | 11 +-
.../physical-plan/src/coalesce_partitions.rs | 12 +-
datafusion/physical-plan/src/coop.rs | 10 +-
datafusion/physical-plan/src/empty.rs | 28 +-
datafusion/physical-plan/src/execution_plan.rs | 58 +-
datafusion/physical-plan/src/explain.rs | 10 +-
datafusion/physical-plan/src/filter.rs | 168 +-
datafusion/physical-plan/src/filter_pushdown.rs | 1 +
datafusion/physical-plan/src/joins/cross_join.rs | 16 +-
.../physical-plan/src/joins/hash_join/exec.rs | 257 ++-
.../physical-plan/src/joins/hash_join/mod.rs | 1 +
.../src/joins/hash_join/partitioned_hash_eval.rs | 158 ++
.../src/joins/hash_join/shared_bounds.rs | 409 +++--
.../physical-plan/src/joins/hash_join/stream.rs | 112 +-
.../physical-plan/src/joins/nested_loop_join.rs | 332 +++-
.../src/joins/piecewise_merge_join/classic_join.rs | 20 +-
.../src/joins/sort_merge_join/exec.rs | 15 +-
.../src/joins/sort_merge_join/metrics.rs | 8 -
.../src/joins/sort_merge_join/stream.rs | 66 +-
.../src/joins/sort_merge_join/tests.rs | 234 +++
.../physical-plan/src/joins/stream_join_utils.rs | 6 -
.../physical-plan/src/joins/symmetric_hash_join.rs | 51 +-
datafusion/physical-plan/src/joins/test_utils.rs | 6 +-
datafusion/physical-plan/src/joins/utils.rs | 62 +-
datafusion/physical-plan/src/lib.rs | 3 +
datafusion/physical-plan/src/limit.rs | 20 +-
datafusion/physical-plan/src/memory.rs | 48 +-
datafusion/physical-plan/src/metrics/baseline.rs | 14 +
datafusion/physical-plan/src/metrics/builder.rs | 22 +-
datafusion/physical-plan/src/metrics/mod.rs | 5 +-
datafusion/physical-plan/src/metrics/value.rs | 300 +++-
datafusion/physical-plan/src/placeholder_row.rs | 14 +-
datafusion/physical-plan/src/projection.rs | 95 +-
datafusion/physical-plan/src/recursive_query.rs | 17 +-
datafusion/physical-plan/src/repartition/mod.rs | 147 +-
datafusion/physical-plan/src/sorts/sort.rs | 43 +-
.../src/sorts/sort_preserving_merge.rs | 14 +-
datafusion/physical-plan/src/sorts/stream.rs | 13 +-
.../physical-plan/src/sorts/streaming_merge.rs | 11 +-
datafusion/physical-plan/src/spill/mod.rs | 1 +
datafusion/physical-plan/src/test.rs | 19 +-
datafusion/physical-plan/src/topk/mod.rs | 67 +-
datafusion/physical-plan/src/union.rs | 52 +-
datafusion/physical-plan/src/unnest.rs | 15 +-
.../physical-plan/src/windows/window_agg_exec.rs | 16 +-
datafusion/physical-plan/src/work_table.rs | 14 +-
datafusion/proto-common/src/lib.rs | 3 +
datafusion/proto/src/common.rs | 14 +-
datafusion/proto/src/lib.rs | 3 +
datafusion/proto/src/logical_plan/mod.rs | 13 +-
datafusion/proto/src/physical_plan/from_proto.rs | 58 +-
datafusion/proto/src/physical_plan/mod.rs | 41 +-
datafusion/proto/src/physical_plan/to_proto.rs | 3 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 4 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 131 +-
datafusion/pruning/src/lib.rs | 4 +
datafusion/pruning/src/pruning_predicate.rs | 22 +-
datafusion/session/src/lib.rs | 4 +
datafusion/spark/src/function/aggregate/avg.rs | 35 +-
datafusion/spark/src/function/bitwise/bit_count.rs | 25 +-
datafusion/spark/src/function/bitwise/bit_get.rs | 237 +--
datafusion/spark/src/function/bitwise/bit_shift.rs | 727 ++-------
datafusion/spark/src/function/bitwise/mod.rs | 18 +-
datafusion/spark/src/function/datetime/date_add.rs | 8 +-
datafusion/spark/src/function/datetime/last_day.rs | 10 +-
.../spark/src/function/datetime/make_interval.rs | 43 +-
datafusion/spark/src/function/hash/crc32.rs | 56 +-
datafusion/spark/src/function/hash/sha1.rs | 58 +-
datafusion/spark/src/function/math/abs.rs | 378 +++++
datafusion/spark/src/function/math/expm1.rs | 63 +-
datafusion/spark/src/function/math/factorial.rs | 10 +-
datafusion/spark/src/function/math/hex.rs | 13 +-
datafusion/spark/src/function/math/mod.rs | 11 +
datafusion/spark/src/function/math/modulus.rs | 28 +-
datafusion/spark/src/function/math/rint.rs | 7 +-
datafusion/spark/src/function/math/trigonometry.rs | 167 ++
.../spark/src/function/string/format_string.rs | 25 +-
datafusion/spark/src/function/string/length.rs | 16 +-
datafusion/spark/src/lib.rs | 3 +
datafusion/sql/src/cte.rs | 12 +-
datafusion/sql/src/expr/binary_op.rs | 4 +-
datafusion/sql/src/expr/function.rs | 93 +-
datafusion/sql/src/expr/identifier.rs | 9 +-
datafusion/sql/src/expr/mod.rs | 16 +-
datafusion/sql/src/expr/subquery.rs | 8 +-
datafusion/sql/src/expr/value.rs | 22 +-
datafusion/sql/src/lib.rs | 3 +
datafusion/sql/src/parser.rs | 103 +-
datafusion/sql/src/resolve.rs | 1 +
datafusion/sql/src/statement.rs | 87 +-
datafusion/sql/src/unparser/expr.rs | 35 +-
datafusion/sql/src/unparser/plan.rs | 20 +-
datafusion/sql/src/unparser/utils.rs | 26 +-
datafusion/sql/src/utils.rs | 11 +-
datafusion/sql/tests/cases/params.rs | 2 +-
datafusion/sql/tests/sql_integration.rs | 24 +-
datafusion/sqllogictest/README.md | 11 +
datafusion/sqllogictest/bin/sqllogictests.rs | 84 +-
.../src/engines/currently_executed_sql.rs | 85 +
.../src/engines/datafusion_engine/runner.rs | 25 +-
.../runner.rs | 20 +
datafusion/sqllogictest/src/engines/mod.rs | 3 +
.../src/engines/postgres_engine/mod.rs | 37 +-
datafusion/sqllogictest/src/lib.rs | 1 +
datafusion/sqllogictest/src/util.rs | 49 +-
.../test_files/agg_func_substitute.slt | 6 +-
datafusion/sqllogictest/test_files/aggregate.slt | 166 +-
.../test_files/aggregate_repartition.slt | 10 +-
.../sqllogictest/test_files/aggregates_topk.slt | 49 +-
datafusion/sqllogictest/test_files/array.slt | 42 +-
datafusion/sqllogictest/test_files/arrow_files.slt | 260 +++
datafusion/sqllogictest/test_files/async_udf.slt | 11 +-
datafusion/sqllogictest/test_files/case.slt | 20 +
.../sqllogictest/test_files/count_star_rule.slt | 23 +-
datafusion/sqllogictest/test_files/cte.slt | 39 +-
datafusion/sqllogictest/test_files/decimal.slt | 139 ++
datafusion/sqllogictest/test_files/dictionary.slt | 15 +-
datafusion/sqllogictest/test_files/explain.slt | 23 +-
.../drop_tables.slt.part => explain_analyze.slt} | 29 +-
.../sqllogictest/test_files/explain_tree.slt | 871 +++++-----
datafusion/sqllogictest/test_files/expr.slt | 207 +++
.../test_files/filter_without_sort_exec.slt | 42 +-
datafusion/sqllogictest/test_files/functions.slt | 19 +-
datafusion/sqllogictest/test_files/group_by.slt | 40 +-
.../sqllogictest/test_files/information_schema.slt | 10 +
datafusion/sqllogictest/test_files/insert.slt | 15 +-
.../sqllogictest/test_files/insert_to_external.slt | 10 +-
datafusion/sqllogictest/test_files/join.slt.part | 37 +-
.../test_files/join_disable_repartition_joins.slt | 9 +-
.../test_files/join_is_not_distinct_from.slt | 9 +-
datafusion/sqllogictest/test_files/joins.slt | 188 +--
datafusion/sqllogictest/test_files/limit.slt | 14 +-
datafusion/sqllogictest/test_files/map.slt | 25 +-
.../test_files/monotonic_projection_test.slt | 10 +-
.../sqllogictest/test_files/named_arguments.slt | 2 +-
datafusion/sqllogictest/test_files/operator.slt | 40 +-
datafusion/sqllogictest/test_files/options.slt | 11 +-
datafusion/sqllogictest/test_files/order.slt | 52 +-
datafusion/sqllogictest/test_files/parquet.slt | 28 +-
.../test_files/parquet_filter_pushdown.slt | 42 +-
.../sqllogictest/test_files/parquet_statistics.slt | 21 +-
datafusion/sqllogictest/test_files/predicates.slt | 49 +-
datafusion/sqllogictest/test_files/prepare.slt | 22 +-
datafusion/sqllogictest/test_files/projection.slt | 7 +-
.../sqllogictest/test_files/push_down_filter.slt | 63 +-
datafusion/sqllogictest/test_files/pwmj.slt | 46 +-
datafusion/sqllogictest/test_files/qualify.slt | 71 +-
.../sqllogictest/test_files/regexp/regexp_like.slt | 5 +-
datafusion/sqllogictest/test_files/repartition.slt | 14 +-
.../sqllogictest/test_files/repartition_scan.slt | 30 +-
datafusion/sqllogictest/test_files/scalar.slt | 2 +-
datafusion/sqllogictest/test_files/select.slt | 44 +-
.../sqllogictest/test_files/set_variable.slt | 171 ++
.../sqllogictest/test_files/simplify_expr.slt | 21 +-
.../{spark/string/concat.slt => slt_features.slt} | 56 +-
.../sqllogictest/test_files/sort_merge_join.slt | 49 +
.../test_files/spark/bitwise/bit_count.slt | 15 +
.../test_files/spark/bitwise/bit_get.slt | 25 +
.../test_files/spark/bitwise/shiftright.slt | 115 ++
.../spark/bitwise/shiftrightunsigned.slt | 103 ++
.../sqllogictest/test_files/spark/hash/crc32.slt | 31 +-
.../sqllogictest/test_files/spark/hash/sha1.slt | 25 +-
.../sqllogictest/test_files/spark/math/abs.slt | 69 +-
.../sqllogictest/test_files/spark/math/csc.slt | 23 +-
.../sqllogictest/test_files/spark/math/expm1.slt | 5 +
.../sqllogictest/test_files/spark/math/sec.slt | 22 +-
.../test_files/spark/math/shiftleft.slt | 100 ++
.../test_files/string/string_literal.slt | 4 +-
datafusion/sqllogictest/test_files/subquery.slt | 32 +-
datafusion/sqllogictest/test_files/timestamps.slt | 55 +-
datafusion/sqllogictest/test_files/topk.slt | 2 +-
.../sqllogictest/test_files/tpch/plans/q1.slt.part | 5 +-
.../test_files/tpch/plans/q10.slt.part | 30 +-
.../test_files/tpch/plans/q11.slt.part | 60 +-
.../test_files/tpch/plans/q12.slt.part | 11 +-
.../test_files/tpch/plans/q13.slt.part | 14 +-
.../test_files/tpch/plans/q14.slt.part | 12 +-
.../test_files/tpch/plans/q15.slt.part | 31 +-
.../test_files/tpch/plans/q16.slt.part | 18 +-
.../test_files/tpch/plans/q17.slt.part | 19 +-
.../test_files/tpch/plans/q18.slt.part | 30 +-
.../test_files/tpch/plans/q19.slt.part | 18 +-
.../sqllogictest/test_files/tpch/plans/q2.slt.part | 103 +-
.../test_files/tpch/plans/q20.slt.part | 64 +-
.../test_files/tpch/plans/q21.slt.part | 49 +-
.../test_files/tpch/plans/q22.slt.part | 26 +-
.../sqllogictest/test_files/tpch/plans/q3.slt.part | 25 +-
.../sqllogictest/test_files/tpch/plans/q4.slt.part | 14 +-
.../sqllogictest/test_files/tpch/plans/q5.slt.part | 45 +-
.../sqllogictest/test_files/tpch/plans/q6.slt.part | 5 +-
.../sqllogictest/test_files/tpch/plans/q7.slt.part | 49 +-
.../sqllogictest/test_files/tpch/plans/q8.slt.part | 61 +-
.../sqllogictest/test_files/tpch/plans/q9.slt.part | 39 +-
datafusion/sqllogictest/test_files/union.slt | 48 +-
datafusion/sqllogictest/test_files/unnest.slt | 6 +-
datafusion/sqllogictest/test_files/window.slt | 177 +--
.../sqllogictest/test_files/window_limits.slt | 10 +-
datafusion/substrait/src/extensions.rs | 4 +-
datafusion/substrait/src/lib.rs | 3 +
.../src/logical_plan/consumer/expr/mod.rs | 4 +-
.../logical_plan/producer/substrait_producer.rs | 8 +-
datafusion/substrait/src/physical_plan/consumer.rs | 4 +-
.../tests/cases/roundtrip_physical_plan.rs | 34 +-
datafusion/wasmtest/src/lib.rs | 3 +
dev/changelog/51.0.0.md | 717 +++++++++
dev/release/README.md | 340 ++--
dev/release/add-branch-protection.sh | 160 ++
dev/update_function_docs.sh | 30 +
docs/requirements.txt | 2 +-
.../contributor-guide/development_environment.md | 2 +-
docs/source/contributor-guide/index.md | 8 +-
docs/source/contributor-guide/testing.md | 32 +-
docs/source/download.md | 26 +-
docs/source/library-user-guide/catalogs.md | 2 +-
docs/source/library-user-guide/upgrading.md | 229 ++-
docs/source/user-guide/configs.md | 237 +--
docs/source/user-guide/expressions.md | 4 +-
docs/source/user-guide/metrics.md | 1 +
docs/source/user-guide/sql/aggregate_functions.md | 30 +
docs/source/user-guide/sql/explain.md | 18 +-
572 files changed, 23502 insertions(+), 10997 deletions(-)
rename datafusion-examples/examples/{ =>
custom_data_source}/csv_json_opener.rs (89%)
rename datafusion-examples/examples/{ =>
custom_data_source}/csv_sql_streaming.rs (96%)
rename datafusion-examples/examples/{ =>
custom_data_source}/custom_datasource.rs (98%)
rename datafusion-examples/examples/{ =>
custom_data_source}/custom_file_casts.rs (98%)
rename datafusion-examples/examples/{ =>
custom_data_source}/custom_file_format.rs (94%)
rename datafusion-examples/examples/{ =>
custom_data_source}/default_column_values.rs (91%)
rename datafusion-examples/examples/{ =>
custom_data_source}/file_stream_provider.rs (91%)
create mode 100644 datafusion-examples/examples/custom_data_source/main.rs
rename datafusion-examples/examples/{ => data_io}/catalog.rs (98%)
rename datafusion-examples/examples/{ => data_io}/json_shredding.rs (99%)
create mode 100644 datafusion-examples/examples/data_io/main.rs
rename datafusion-examples/examples/{advanced_parquet_index.rs =>
data_io/parquet_advanced_index.rs} (98%)
rename datafusion-examples/examples/{ => data_io}/parquet_embedded_index.rs
(98%)
rename datafusion-examples/examples/{ => data_io}/parquet_encrypted.rs (96%)
rename datafusion-examples/examples/{ =>
data_io}/parquet_encrypted_with_kms.rs (99%)
rename datafusion-examples/examples/{ => data_io}/parquet_exec_visitor.rs (93%)
rename datafusion-examples/examples/{ => data_io}/parquet_index.rs (99%)
rename datafusion-examples/examples/{query-http-csv.rs =>
data_io/query_http_csv.rs} (91%)
rename datafusion-examples/examples/{ => data_io}/remote_catalog.rs (98%)
rename datafusion-examples/examples/{ => dataframe}/dataframe.rs (97%)
rename datafusion-examples/examples/{ => dataframe}/deserialize_to_struct.rs
(98%)
copy datafusion-examples/examples/{builtin_functions => dataframe}/main.rs
(63%)
copy datafusion-examples/examples/{builtin_functions =>
execution_monitoring}/main.rs (53%)
rename datafusion-examples/examples/{ =>
execution_monitoring}/memory_pool_execution_plan.rs (98%)
rename datafusion-examples/examples/{ =>
execution_monitoring}/memory_pool_tracking.rs (95%)
rename datafusion-examples/examples/{ => execution_monitoring}/tracing.rs (97%)
rename datafusion-examples/examples/{ => proto}/composed_extension_codec.rs
(95%)
copy datafusion-examples/examples/{builtin_functions => proto}/main.rs (63%)
rename datafusion-examples/examples/{ => query_planning}/analyzer_rule.rs (98%)
rename datafusion-examples/examples/{ => query_planning}/expr_api.rs (99%)
create mode 100644 datafusion-examples/examples/query_planning/main.rs
rename datafusion-examples/examples/{ => query_planning}/optimizer_rule.rs
(98%)
rename datafusion-examples/examples/{ => query_planning}/parse_sql_expr.rs
(96%)
rename datafusion-examples/examples/{ => query_planning}/plan_to_sql.rs (95%)
rename datafusion-examples/examples/{ => query_planning}/planner_api.rs (98%)
rename datafusion-examples/examples/{ => query_planning}/pruning.rs (97%)
rename datafusion-examples/examples/{ => query_planning}/thread_pools.rs (99%)
rename datafusion-examples/examples/{sql_analysis.rs => sql_ops/analysis.rs}
(98%)
rename datafusion-examples/examples/{sql_dialect.rs => sql_ops/dialect.rs}
(98%)
rename datafusion-examples/examples/{sql_frontend.rs => sql_ops/frontend.rs}
(99%)
copy datafusion-examples/examples/{builtin_functions => sql_ops}/main.rs (59%)
rename datafusion-examples/examples/{sql_query.rs => sql_ops/query.rs} (99%)
delete mode 100644 datafusion/common/src/pyarrow.rs
copy datafusion/core/tests/{catalog => catalog_listing}/mod.rs (96%)
create mode 100644
datafusion/core/tests/catalog_listing/pruned_partition_list.rs
create mode 100644
datafusion/core/tests/data/partitioned_table_arrow_stream/part=123/data.arrow
create mode 100644
datafusion/core/tests/data/partitioned_table_arrow_stream/part=456/data.arrow
create mode 100644 datafusion/core/tests/execution/register_arrow.rs
create mode 100644
datafusion/core/tests/user_defined/user_defined_async_scalar_functions.rs
create mode 100644 datafusion/datasource-arrow/tests/data/example_stream.arrow
create mode 100644
datafusion/datasource-arrow/tests/data/example_stream_corrupted_metadata_length.arrow
create mode 100644
datafusion/datasource-arrow/tests/data/example_stream_empty.arrow
create mode 100644
datafusion/expr-common/src/type_coercion/binary/tests/run_end_encoded.rs
create mode 100644 datafusion/expr/src/expr_rewriter/guarantees.rs
create mode 100644 datafusion/expr/src/predicate_bounds.rs
create mode 100644 datafusion/ffi/src/catalog_provider_list.rs
copy datafusion/ffi/tests/{ffi_integration.rs => ffi_catalog.rs} (50%)
create mode 100644
datafusion/functions-aggregate-common/src/noop_accumulator.rs
delete mode 100644 datafusion/optimizer/src/eliminate_one_union.rs
rename datafusion/optimizer/src/{eliminate_nested_union.rs =>
optimize_unions.rs} (90%)
delete mode 100644 datafusion/optimizer/src/simplify_expressions/guarantees.rs
delete mode 100644
datafusion/physical-optimizer/src/coalesce_async_exec_input.rs
create mode 100644
datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs
create mode 100644 datafusion/spark/src/function/math/abs.rs
create mode 100644 datafusion/spark/src/function/math/trigonometry.rs
create mode 100644
datafusion/sqllogictest/src/engines/currently_executed_sql.rs
copy datafusion/sqllogictest/test_files/{tpch/drop_tables.slt.part =>
explain_analyze.slt} (69%)
copy datafusion/sqllogictest/test_files/{spark/string/concat.slt =>
slt_features.slt} (56%)
create mode 100644 dev/changelog/51.0.0.md
create mode 100755 dev/release/add-branch-protection.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
