This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/checkout-6.0.1
in repository https://gitbox.apache.org/repos/asf/datafusion-sandbox.git
discard 60418e18a chore(deps): bump actions/checkout from 5.0.0 to 6.0.1
add c2ba08717 Avoid skew in Roundrobin repartition (#18880)
add 76b9e123e Limit visibility of internal impl functions in function
crates (#18877)
add 3230048fa Add benchmark for array_has/array_has_all/array_has_any
(#18729)
add d041ba2ab chore(deps): bump maturin from 1.10.0 to 1.10.2 in /docs
(#18905)
add 91bd6966e chore(deps): bump taiki-e/install-action from 2.62.54 to
2.62.56 (#18899)
add faca8d7e9 chore(deps): bump indicatif from 0.18.0 to 0.18.3 (#18897)
add 710d14b0d chore(deps): bump tokio-util from 0.7.16 to 0.7.17 (#18898)
add 1cc9bcd52 FFI: return underlying trait type when converting from FFI
structs (#18672)
add 2dbd9c7e4 Support Non-Literal Expressions in Substrait VirtualTable
Values and Improve Round-Trip Robustness (#18866)
add 5c24b80cb chore(deps): bump indexmap from 2.12.0 to 2.12.1 (#18895)
add ec51b94ee chore(deps): bump aws-config from 1.8.7 to 1.8.11 (#18896)
add d24eb4a23 chore(deps): bump flate2 from 1.1.4 to 1.1.5 (#18900)
add 6856dc423 Add iter() method to `Extensions` (#18887)
add 6ea305ec4 feat: allow custom caching via logical node (#18688)
add 2db3aeaa5 chore: Enforce `clippy::needless_pass_by_value` globally
across the workspace (#18904)
add 39d4a3876 Consolidate external dependency examples (#18142) (#18747)
add c83b9d722 feat: add `array_slice` benchmark (#18879)
add 48cc4c8af Optimize planning for projected nested union (#18713)
add 838e1dea8 Update links in documentation to point at new example
locations (#18931)
add d5f1149bf chore(deps): bump taiki-e/install-action from 2.62.56 to
2.62.57 (#18927)
add c7b6afd20 chore(deps): bump actions/setup-python from 6.0.0 to 6.1.0
(#18925)
add 10706aeb3 Fix `map` function alias handling in SQL planner (#18914)
add 3ba7350d5 feat: Support recursive queries with a distinct 'UNION'
(#18254)
add 7cbb4432b minor: add builder setting
`NdJsonReadOptions::schema_infer_max_records` (#18920)
add 14f34f641 fix: preserve byte-size statistics in AggregateExec (#18885)
add 087174102 Add Kubeflow Trainer to known users (#18935)
add 8ab83425c Implement Substrait Support for `GROUPING SET CUBE` (#18798)
add 0366f0ef5 chore: unify common dependencies as workspace dependencies
(#18665)
add 54a78689c Fix bug where binary types were incorrectly being casted for
coercible signatures (#18750)
add 211877f1a Refactor approx_median signature & support f16 (#18647)
add 473cbdab8 Refactor `to_local_time()` signature away from user_defined
(#18707)
add 4eb293344 Refactor crypto functions code (#18664)
add 769f36732 fix: Track elapsed_compute metric for CSV scans (#18901)
add 8615ef4a9 chore(deps-dev): bump node-forge from 1.3.1 to 1.3.2 in
/datafusion/wasmtest/datafusion-wasm-app (#18958)
add 9f725d9c7 move projection handling into FileSource (#18627)
add b990987bd Add PhysicalOptimizerRule::optimize_plan to allow passing
more context into optimizer rules (#18739)
add c6f73636b Support LikeMatch, ILikeMatch, NotLikeMatch, NotILikeMatch
operators in protobuf serialization (#18961)
add fb14d7cb1 fix: Implement Substrait consumer support for like_match,
like_imatch, and negated variants (#18929)
add cb218dc38 chore: cargo fmt to fix CI (#18969)
add 7516b642a chore(deps): bump Swatinem/rust-cache from 2.8.1 to 2.8.2
(#18963)
add 53a89b20e chore(deps): bump taiki-e/install-action from 2.62.57 to
2.62.58 (#18964)
add 70edcc1f3 chore(deps): bump crate-ci/typos from 1.39.2 to 1.40.0
(#18965)
add 02a7181a7 [Minor] Refactor `traverse_chain` macro to function (#18951)
add 525835280 fix: Initialize CsvOptions::double_quote from
proto_opts.double_quote (#18967)
add 79257353b Enable clippy::allow_attributes lint for datafusion-catalog
(#18973)
add 36ec9f1de Feat/update rust 2024 edition (#18915)
add 487de9db6 chore(deps): bump taiki-e/install-action from 2.62.58 to
2.62.59 (#18978)
add 73562e8ea Simplify percentile_cont for 0/1 percentiles (#18837)
add 1f03ac502 chore: enforce clippy::allow_attributes for functions-*
crates (#18986)
add 2bfa64d28 chore: enforce clippy::allow_attributes for common crates
(#18988)
add e54eb4238 Fix predicate_rows_pruned & predicate_rows_matched metrics
(#18980)
add 4cfe20ffe Allocate a buffer of the correct length for
ScalarValue::FixedSizeBinary in ScalarValue::to_array_of_size (#18903)
add e4dcf0c85 Fix error planning aggregates with duplicated names in
select list (#18831)
add 96a5f21c4 optimizer: Support dynamic filter in `MIN/MAX` aggregates
(#18644)
add 81512da2b feat: Makes error macros hygienic (#18995)
add 1a9821e30 chore: remove `deny`s of `needless_pass_by_value` in
`lib.rs` files (#18996)
add c89cb7078 Add Explicit Error Handling for Unsupported SQL `FETCH`
Clause in Planner and CLI (#18691)
add 033ce4301 chore(deps): bump criterion from 0.7.0 to 0.8.0 (#19009)
add a3f1250d7 chore(deps): bump syn from 2.0.108 to 2.0.111 (#19011)
add 6425e40c3 chore(deps): bump taiki-e/install-action from 2.62.59 to
2.62.60 (#19012)
add 901aacaec chore: remove redundant clone code (#18997)
add 8a999e189 fix: `rstest` is a DEV dependency (#19014)
add a06073913 Update to `arrow`, `parquet` to `57.1.0` (#18820)
add 0f1133e9f fix: partition pruning stats pruning when multiple values
are present (#18923)
add 3b6e2fbfa fix: deprecate data_type_and_nullable and simplify API usage
(#18869)
add bc47cbe73 Add PGO documentation section to crate configuration (#18959)
add 5519b6145 deny on allow_attributes lint in physical-plan (#18983)
add 477053d83 Optimize planning / stop cloning Strings / Fields so much
(2-3% faster planning time) (#18415)
add e43e8bf13 Add additional test coverage of multi-value
PartitionPruningStats (#19021)
add fa3048969 perf: use `new_repeated` when converting scalar to an array
(#19018)
add 2c24deb0e Fix tpch benchmark harness (#19033)
add da36ad83c Fix data for tpch_csv and tpch_csv10 (#19034)
add ed836d739 chore: update group of 3 crates to rust 2024 edition (#19001)
add cab28a6a7 chore(deps-dev): bump express from 4.21.2 to 4.22.1 in
/datafusion/wasmtest/datafusion-wasm-app (#19040)
add 7b4593f36 Add upgrade guide for PhysicalOptimizerRule::optimize_plan
(#19030)
add a43f33db5 Allow repartitioning on files with ranges (#18948)
add 0490aec7e Support simplify not for physical expr (#18970)
add a1bb74b40 dev: Add typos check to the local `dev/rust_lint.sh` (#17863)
add 653caa143 doc: add `FilterExec` metrics to `user-guide/metrics.md`
(#19043)
add 5842e23c4 Implement FFI_PhysicalExpr and the structs it needs to
support it. (#18916)
add 4d86ae091 fix: pre-warm listing file statistics cache during listing
table creation (#18971)
add 9ee687276 chore(deps): bump actions/setup-node from 6.0.0 to 6.1.0
(#19063)
add e30ca2e45 chore(deps): bump taiki-e/install-action from 2.62.60 to
2.62.61 (#19062)
add 09fe307ca chore(deps): bump actions/stale from 10.1.0 to 10.1.1
(#19061)
add 2217c2b83 chore: merge make_array and spark array (#19006)
add b402dff94 chore(deps): bump actions/checkout from 6.0.0 to 6.0.1
(#19060)
add 11c9789b0 Add documentation example for `PartitionPruningStatistics`
(#19020)
add 9af6858ad Add `force_filter_selections` to restore `pushdown_filters`
behavior prior to parquet 57.1.0 upgrade (#19003)
add 91e623bf8 chore: upgrade expr and execution crates to rust 2024
edition (#19047)
add d4820d171 fix: log metadata differences when comparing physical and
logical schema (#19070)
add 0c6b65414 Implement FFI task context and task context provider (#18918)
add 6751f441f refactor: Refactor spark make_interval signature away from
user defined (#19027)
add 434a23b1b Fix: Align sort_merge_join filter output with join schema to
fix right-anti panic (#18800)
add 90aa47e19 Support Substrait Round-Trip of `EmptyRelation` Including
`produce_one_row` Semantics (#18842)
add 00e79523b chore(deps): bump taiki-e/install-action from 2.62.61 to
2.62.62 (#19081)
add 71fdad080 chore: enforce clippy::allow_attributes for datasource
crates (#19068)
add 63a8c659e common: Add hashing support for REE arrays (#18981)
add f22a3f395 Use `tpchgen-cli` to generate tpch data in bench.sh (#19035)
add 8dac8f1c4 Minor: fix link errors in docs (#19088)
add 388db0e52 Update aggregate probe to be locked only if skipping
aggregation (#18766)
add d150af7b2 Adds memory-bound DefaultListFilesCache (#18855)
add 107cb5e8f Fix function doc CI check (#19093)
add a3a020f6b Fix Schema Duplication Errors in Self‑Referential
INTERSECT/EXCEPT by Requalifying Input Sides (#18814)
add e01fa0ce4 Cut `Parquet` over to PhysicalExprAdapter, remove
`SchemaAdapter` (#18998)
add a33ae5e57 run cargo fmt to fix after #18998 (#19102)
add a5fc3c704 bench: set test_util as required feature for
aggregate_vectorized (#19101)
add 1844c88ed Update Committer / PMC list (#19105)
add 03c1a8df9 use ProjectionExprs:project_statistics in FileScanConfig
(#19094)
add f04a0eb26 Temporarily ignore test_cache_with_ttl_and_lru test (#19115)
add 482c6b8bd refactor: move human readable display utilities to
`datafusion-common` crate (#19080)
add 2b05b09b6 feat: Add builder API for CreateExternalTable to reduce
verbosity (#19066)
add 1800d150f Always remove unecessary software from github runners for
all jobs (fix intermittent out of space on runners) (#19122)
add c293854a3 fix: fix panic when lo is greater than hi (#19099)
add 340a28ca6 fix: escape underscores when simplifying `starts_with`
(#19077)
add cf9d07851 [datafusion-spark]: Refactor make_dt_interval's signature
away from user defined (#19083)
add 674600782 feat(spark): Implement Spark functions `url_encode`,
`url_decode` and `try_url_decode` (#17399)
add c479deeb6 fix deprecation notes with incorrect versions from #13083
(#19135)
add 944f7f2f2 Run the examples in the new format (#18946)
add cb2f3d2a8 Add constant expression evaluator to physical expression
simplifier (#19130)
add fc6d0a428 Fix shuffle function to report nullability correctly (#19184)
add 41f713758 chore: enforce clippy::allow_attributes for physical crates
(#19185)
add 3a0064d82 fix: custom nullability for length (#19175) (#19182)
add 998f534aa Update 5 crates to rust 2024 edition (#19091)
add 215cdffb6 Coalesce batches inside hash join, reuse indices buffer
(#18972)
add dd35f7c65 slt test coverage for `CASE` exprs with constant value
lookup tables (#19143)
add 4c3e3c1c6 Fix fmt after logical conflict (#19208)
add e8384fbbf chore: Add TPCDS benchmarks (#19138)
add adaed426a Arc partition values in TableSchema (#19137)
add 0812d2731 feat: Move DefaultMetadataCache into its own module (#19125)
add 662a3bad6 fix: inverted null_percent logic in in_list benchmark
(#19204)
add cde6dfab1 Add sorted data benchmark. (#19042)
add f2b476f39 Refactor PhysicalExprSimplfier to &self instead of &mut self
(#19212)
add 3fabee770 Revert adding PhysicalOptimizerRule::optimize_plan (#19186)
add 4504d941b chore(deps): bump uuid from 1.18.1 to 1.19.0 (#19199)
add af3ab52a7 chore(deps): bump async-compression from 0.4.34 to 0.4.35
(#19201)
add dcc09c345 chore(deps): bump taiki-e/install-action from 2.62.62 to
2.62.63 (#19198)
add 7d958b040 chore(deps): bump tracing-subscriber from 0.3.20 to 0.3.22
(#19200)
add 71a698285 chore(deps): bump wasm-bindgen-test from 0.3.55 to 0.3.56
(#19202)
add 55a38d4a6 feat: Add `remove_optimizer_rule` to `SessionContext`
(#19209)
add ab7fe0eb5 bench: add dedicated Utf8View benchmarks for InList (#19211)
add 83736efc4 Fix PruningPredicate interaction with
DynamicFilterPhysicalExpr that references partition columns (#19129)
add ad9b779b6 Implement physical and logical codecs in FFI (#19079)
add 4ddee14ee refactor: Refactor spark width bucket signature away from
user defined (#19065)
add 2626fc202 fix: Ensure column names do not change with
`expand_views_at_output` (#19019)
add 4fb36b23d fix: bitmap_count should report nullability correctly
(#19195)
add 1a6df66e2 fix: bit_count function to report nullability correctly
(#19197)
add ea9f35412 fix: derive custom nullability for spark `bit_shift` (#19222)
add 7ea5066fe Sort Merge Join: Reduce batch concatenation, use
`BatchCoalescer`, new benchmarks (TPC-H Q21 SMJ up to ~4000x faster) (#18875)
add a30cf3709 Add relation planner extension support to customize SQL
planning (#17843)
add 20870c18a feat: integrate batch coalescer with repartition exec
(#19002)
add c0e8bb501 Push down InList or hash table references from HashJoinExec
depending on the size of the build side (#18393)
add 21a16e44b Add additional tests for InListExpr (#19050)
add 7ac40d796 chore(deps): bump taiki-e/install-action from 2.62.63 to
2.62.64 (#19226)
add dc786138c Move partition handling out of PhysicalExprAdapter (#19128)
add cd160de0b fix: spark elt custom nullability (#19207)
add dc6a71235 Use strum in the examples (#19126) (#19205)
add d85595e03 perf: optimize CASE WHEN lookup table (2.5-22.5 times
faster) (#18183)
add e8a0829b7 Allow Logical expression ScalarVariable to represent an
extension type or metadata (#18243)
add dc4e3ab47 feat: Implement the `statistics_cache` function (#19054)
add ca67edcf9 [Proto]: Serialization support for `AsyncFuncExec` (#19118)
add 26af02a48 chore: add test case for decimal overflow (#19255)
add 79a2f5e11 chore(deps): bump taiki-e/install-action from 2.62.64 to
2.62.65 (#19251)
add 2a08013af chore: update 6 crates to rust edition 2024 (#19196)
add 021188e71 Push down projection expressions into ParquetOpener (#19111)
add 321401ce6 fix: `array_remove`/`array_remove_n`/`array_remove_all` not
using the same nullability as the input (#19259)
add c1aa1b530 Track column sizes in Statistics; propagate through
projections (#19113)
add 39a1f7575 Implement FFI_Session (#19223)
add 89a9c9d06 Feat: Add an option for fast tests by gating slow tests to
extended_tests feature (#19237)
add 5496c3043 fix: typo in sql/ddl (#19276)
add e91493566 chore: enforce clippy::allow_attributes for 7 crates (#19133)
add bde16083a feat: Preserve File Partitioning From File Scans (#19124)
add 58345f94a Improve ProjectionExpr documentation and comments (#19263)
add 85d8a88c1 add specialized InList implementations for common scalar
types (#18832)
add c8add5c1d Update README occording to the new examples (#18529) (#19257)
add 044a4a736 Add make_time function (#19183)
add 33b3f0216 dev: Add CI doc prettier check to local `rust_lint.sh`
(#19254)
add a3b3eb558 bug: Eliminate dead round-robin insertion in enforce
distribution (#19132)
add f9c030a5f Automatically download tpcds benchmark data to the right
place (#19244)
add 86011519b fix: flaky cache test (#19140)
add 10c933cc7 [datafusion-spark]: Refactor hex's signature away from
user_defined (#19235)
add 06f27e94d fix : correct nullability propagation for spark.bitwise_not
(#19224)
add aebae990d fix: Add custom nullability for Spark ILIKE function (#19206)
add 96ddd558f fix: derive custom nullability for spark `map_from_arrays`
(#19275)
add 18e4a0caf fix: derive custom nullability for spark map_from_entries
(#19274)
add 2c6f6d369 fix: derive custom nullable for spark `make_dt_interval`
(#19236)
add c4ca946ca fix: derive custome nullable for the spark last_day (#19232)
add 49cfee05c fix: derive custom nullable for spark `date_sub` (#19225)
add 845894641 added custom nullability for char (#19268)
add 5a01e6864 Update to_date udf function to support a consistent set of
argument types (#19134)
add d61f1a715 replace HashTableLookupExpr with lit(true) in proto
serialization (#19300)
add fedddbcae chore: fix return_field_from_args doc (#19307)
add 4088192fc chore: enforce clippy::allow_attributes for
spark,sql,sustrait (#19309)
add 94d9c8e68 Simplify make_date & fix null handling (#19296)
add 033116e8a fix: Fix a few minor issues with join metrics (#19283)
add 9163feed6 fix: derive nullability for spark `bit_get` (#19220)
add 16fa35fa2 Allow base64 encoding of fixedsizebinary arrays (#18950)
add befaf93a1 chore: update 11 crates to Rust 2024 edition (#19258)
add ec11f4250 feat: Add constant column extraction and rewriting for
projections in ParquetOpener (#19136)
add dbf9265e5 fix: pow() with integer base and negative float exponent
returns error (#19303)
add 2bea79689 Minor: remove unnecessary unit tests for fixed size binary
(#19318)
add 899a76223 Populate partition column statistics for PartitionedFile
(#19284)
add 7c05b208e refactor: move metrics module to `datafusion-common` crate
(#19247)
add 9d4fe1589 Add library user guide for extending SQL syntax (#19265)
add b828fbfa5 chore(deps): bump taiki-e/install-action from 2.62.65 to
2.62.67 (#19295)
add 58377bf2e chore(deps): bump ctor from 0.6.1 to 0.6.3 (#19328)
add fc8824011 Refactor `power()` signature away from user defined (#18968)
add 933657e2c feat: Support sliding window queries for MedianAccumulator
by implementing `retract_batch` (#19278)
add 02c647ae1 chore: enforce `clippy::allow_attributes` for optimizer and
macros (#19310)
add 3a67d86c9 fix(concat): correct nullability inference (nullable only if
all arguments nullable) (#19189)
add 266a7d060 chore(deps): bump taiki-e/install-action from 2.62.67 to
2.63.3 (#19349)
add ea8087d77 chore(deps): bump clap from 4.5.50 to 4.5.53 (#19326)
add 199cf06d2 chore(deps): bump insta from 1.43.2 to 1.44.3 (#19327)
add 79cfe8e92 Add runtime config options for `list_files_cache_limit` and
`list_files_cache_ttl` (#19108)
add 89af69411 remove repartition exec from coalesce batches optimizer
(#19239)
add be0cf05f9 minor: cleanup unnecessary config in `decimal.slt` (#19352)
add c53a44891 Fix panic for `GROUPING SETS(())` and handle empty-grouping
aggregates (#19252)
add 775277ae3 Update datafusion-core crate to Rust 2024 edition (#19332)
add efd793ba3 Minor: clean up titles and links n extending operators and
optimizer pages (#19317)
add 1f2671628 Update 4 crates to rust 2024 edition (#19357)
add 50d20dd55 preserve Field metadata in first_value/last_value (#19335)
add 3a41cc607 Establish the high level API for sort pushdown and the
optimizer rule and support reverse files and row groups (#19064)
add b806117bf Fix flaky SpillPool channel test by synchronizing reader and
writer tasks (#19110)
add 2b67ba81c [minor] Upgrade rust version (#19363)
add 401345b6f Minor: fix cargo fmt (#19368)
add 7900cd6e9 chore: enforce clippy::allow_attributes for proto, pruning,
session (#19350)
add 59dcc362c Update remaining crates to rust 2024 edition (#19361)
add 2e95f2673 Minor: Make `ProjectionExpr::new` easier to use with
constants (#19343)
add fe11ad6b4 fix: Added nullable return from date_add(#19151) (#19229)
add 3879b2450 fix: spark sha1 nullability reporting (#19242)
add 1e4bd7565 Feat: DefaultListFilesCache prefix-aware for partition
pruning optimization (#19298)
add 14cd71eab feat: add compression level configuration for JSON/CSV
writers (#18954)
add 4e7bba490 Extend in_list benchmark coverage (#19376)
add d68b629dc Add hashing microbenchmark `with_hashes` (#19373)
add cb3fa1c2c fix: derive custom nullability for the spark `next_day`
(#19253)
add 0a0416dbf Move `newlines_in_values` from `FileScanConfig` to
`CsvSource` (#19313)
add 2c9077a8b [datafusion-cli] Implement average LIST duration for object
store profiling (#19127)
add 8120da40a chore(deps): bump taiki-e/install-action from 2.63.3 to
2.64.0 (#19382)
add ead820980 update insta snapshots (#19381)
add d59ebac9f feat(spark): implement Spark `try_sum` function (#18569)
add e5c4c9702 Merge branch 'main' into sandbox-main
add aede11c97 chore(deps): bump actions/checkout from 5.0.0 to 6.0.1
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (60418e18a)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/checkout-6.0.1 (aede11c97)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/setup-builder/action.yaml | 14 +
.github/workflows/audit.yml | 2 +-
.github/workflows/dev.yml | 30 +-
.github/workflows/docs.yaml | 2 +-
.github/workflows/docs_pr.yaml | 2 +-
.github/workflows/rust.yml | 55 +-
.github/workflows/stale.yml | 2 +-
Cargo.lock | 492 +++--
Cargo.toml | 33 +-
benchmarks/Cargo.toml | 2 +-
benchmarks/README.md | 143 +-
benchmarks/bench.sh | 283 ++-
.../regen.sh => benchmarks/compare_tpcds.sh | 41 +-
.../regen.sh => benchmarks/compare_tpch.sh | 39 +-
.../clickbench/queries/{ => sorted_data}/q0.sql | 3 +-
benchmarks/src/bin/dfbench.rs | 8 +-
benchmarks/src/bin/external_aggr.rs | 6 +-
benchmarks/src/bin/mem_profile.rs | 27 +-
benchmarks/src/bin/tpch.rs | 65 -
benchmarks/src/cancellation.rs | 10 +-
benchmarks/src/clickbench.rs | 119 +-
benchmarks/src/h2o.rs | 4 +-
benchmarks/src/hj.rs | 2 +-
benchmarks/src/imdb/convert.rs | 2 +-
benchmarks/src/imdb/run.rs | 8 +-
benchmarks/src/lib.rs | 2 +
benchmarks/src/nlj.rs | 6 +-
benchmarks/src/smj.rs | 524 +++++
benchmarks/src/sort_tpch.rs | 4 +-
.../tests/catalog => benchmarks/src/tpcds}/mod.rs | 3 +-
benchmarks/src/tpcds/run.rs | 356 +++
benchmarks/src/tpch/convert.rs | 162 --
benchmarks/src/tpch/mod.rs | 3 -
benchmarks/src/tpch/run.rs | 12 +-
benchmarks/src/util/memory.rs | 2 +-
benchmarks/src/util/options.rs | 2 +-
benchmarks/src/util/run.rs | 2 +-
ci/scripts/doc_prettier_check.sh | 57 +
ci/scripts/rust_example.sh | 30 +-
ci/scripts/{rust_docs.sh => typos_check.sh} | 5 +-
datafusion-cli/Cargo.toml | 4 +-
datafusion-cli/examples/cli-session-context.rs | 2 +-
datafusion-cli/src/catalog.rs | 50 +-
datafusion-cli/src/cli_context.rs | 2 +-
datafusion-cli/src/command.rs | 2 +-
datafusion-cli/src/exec.rs | 27 +-
datafusion-cli/src/functions.rs | 120 +-
datafusion-cli/src/helper.rs | 10 +-
datafusion-cli/src/highlighter.rs | 4 +-
datafusion-cli/src/main.rs | 142 +-
datafusion-cli/src/object_storage.rs | 51 +-
datafusion-cli/src/object_storage/instrumented.rs | 114 +-
datafusion-cli/src/print_format.rs | 66 +-
datafusion-cli/src/print_options.rs | 8 +-
datafusion-cli/tests/cli_integration.rs | 34 +-
...errides@explain_plan_environment_overrides.snap | 1 -
.../tests/snapshots/[email protected] | 2 +-
datafusion-cli/tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
datafusion-cli/tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/cli_quick_test@batch_size.snap | 2 +-
.../cli_quick_test@default_explain_plan.snap | 1 -
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
datafusion-examples/Cargo.toml | 16 +-
datafusion-examples/README.md | 229 +-
.../examples/builtin_functions/date_time.rs | 7 +-
.../examples/builtin_functions/function_factory.rs | 2 +-
.../examples/builtin_functions/main.rs | 73 +-
.../examples/builtin_functions/regexp.rs | 93 +-
.../examples/custom_data_source/csv_json_opener.rs | 30 +-
.../custom_data_source/custom_datasource.rs | 9 +-
.../custom_data_source/custom_file_casts.rs | 31 +-
.../custom_data_source/custom_file_format.rs | 6 +-
.../custom_data_source/default_column_values.rs | 142 +-
.../custom_data_source/file_stream_provider.rs | 6 +-
.../examples/custom_data_source/main.rs | 115 +-
datafusion-examples/examples/data_io/catalog.rs | 4 +-
.../examples/data_io/json_shredding.rs | 114 +-
datafusion-examples/examples/data_io/main.rs | 128 +-
.../examples/data_io/parquet_advanced_index.rs | 16 +-
.../examples/data_io/parquet_embedded_index.rs | 30 +-
.../examples/data_io/parquet_encrypted.rs | 4 +-
.../examples/data_io/parquet_exec_visitor.rs | 23 +-
.../examples/data_io/parquet_index.rs | 12 +-
.../examples/data_io/remote_catalog.rs | 2 +-
.../examples/dataframe/dataframe.rs | 41 +-
datafusion-examples/examples/dataframe/main.rs | 75 +-
.../examples/execution_monitoring/main.rs | 89 +-
.../memory_pool_execution_plan.rs | 3 +-
.../examples/execution_monitoring/tracing.rs | 14 +-
.../{dataframe-to-s3.rs => dataframe_to_s3.rs} | 22 +-
.../examples/external_dependency/main.rs | 84 +
.../{query-aws-s3.rs => query_aws_s3.rs} | 13 +-
.../examples/ffi/ffi_module_interface/Cargo.toml | 2 +-
.../examples/ffi/ffi_module_interface/src/lib.rs | 3 +-
.../examples/ffi/ffi_module_loader/Cargo.toml | 3 +-
.../examples/ffi/ffi_module_loader/src/main.rs | 10 +-
datafusion-examples/examples/flight/main.rs | 80 +-
datafusion-examples/examples/flight/server.rs | 3 +-
datafusion-examples/examples/flight/sql_server.rs | 4 +-
.../examples/proto/composed_extension_codec.rs | 2 +-
datafusion-examples/examples/proto/main.rs | 69 +-
.../examples/query_planning/analyzer_rule.rs | 4 +-
.../examples/query_planning/expr_api.rs | 31 +-
.../examples/query_planning/main.rs | 102 +-
.../examples/query_planning/optimizer_rule.rs | 2 +-
.../examples/query_planning/plan_to_sql.rs | 2 +-
.../examples/query_planning/pruning.rs | 1 +
.../examples/relation_planner/main.rs | 121 ++
.../examples/relation_planner/match_recognize.rs | 406 ++++
.../examples/relation_planner/pivot_unpivot.rs | 567 +++++
.../examples/relation_planner/table_sample.rs | 845 ++++++++
datafusion-examples/examples/sql_ops/analysis.rs | 2 +-
datafusion-examples/examples/sql_ops/frontend.rs | 2 +-
datafusion-examples/examples/sql_ops/main.rs | 77 +-
datafusion-examples/examples/sql_ops/query.rs | 10 +-
datafusion-examples/examples/udf/advanced_udaf.rs | 4 +-
datafusion-examples/examples/udf/advanced_udf.rs | 6 +-
datafusion-examples/examples/udf/advanced_udwf.rs | 49 +-
datafusion-examples/examples/udf/async_udf.rs | 31 +-
datafusion-examples/examples/udf/main.rs | 101 +-
datafusion-examples/examples/udf/simple_udtf.rs | 9 +-
datafusion-examples/examples/udf/simple_udwf.rs | 48 +-
datafusion/catalog-listing/src/config.rs | 2 +-
datafusion/catalog-listing/src/helpers.rs | 23 +-
datafusion/catalog-listing/src/mod.rs | 5 +-
datafusion/catalog-listing/src/options.rs | 4 +-
datafusion/catalog-listing/src/table.rs | 155 +-
datafusion/catalog/src/async.rs | 10 +-
datafusion/catalog/src/catalog.rs | 2 +-
datafusion/catalog/src/default_table_source.rs | 2 +-
datafusion/catalog/src/information_schema.rs | 24 +-
datafusion/catalog/src/lib.rs | 4 +-
datafusion/catalog/src/listing_schema.rs | 23 +-
datafusion/catalog/src/memory/schema.rs | 2 +-
datafusion/catalog/src/memory/table.rs | 6 +-
datafusion/catalog/src/schema.rs | 6 +-
datafusion/catalog/src/stream.rs | 2 +-
datafusion/catalog/src/streaming.rs | 6 +-
datafusion/catalog/src/table.rs | 2 +-
datafusion/catalog/src/view.rs | 2 +-
datafusion/common-runtime/src/common.rs | 8 +-
datafusion/common-runtime/src/lib.rs | 5 +-
datafusion/common-runtime/src/trace_utils.rs | 2 +-
datafusion/common/Cargo.toml | 7 +-
datafusion/common/benches/with_hashes.rs | 209 ++
datafusion/common/src/cast.rs | 8 +-
datafusion/common/src/config.rs | 155 +-
datafusion/common/src/cse.rs | 8 +-
datafusion/common/src/datatype.rs | 114 +-
datafusion/common/src/dfschema.rs | 157 +-
datafusion/common/src/display/human_readable.rs | 139 ++
datafusion/common/src/display/mod.rs | 1 +
datafusion/common/src/error.rs | 77 +-
datafusion/common/src/file_options/csv_writer.rs | 17 +
datafusion/common/src/file_options/json_writer.rs | 18 +-
datafusion/common/src/file_options/mod.rs | 2 +-
.../common/src/file_options/parquet_writer.rs | 42 +-
datafusion/common/src/format.rs | 6 +-
datafusion/common/src/hash_utils.rs | 248 ++-
datafusion/common/src/instant.rs | 2 +-
datafusion/common/src/lib.rs | 32 +-
datafusion/common/src/metadata.rs | 14 +-
datafusion/common/src/nested_struct.rs | 6 +-
datafusion/common/src/param_value.rs | 2 +-
datafusion/common/src/pruning.rs | 145 +-
datafusion/common/src/rounding.rs | 2 +-
datafusion/common/src/scalar/cache.rs | 6 +-
datafusion/common/src/scalar/mod.rs | 238 +-
datafusion/common/src/stats.rs | 285 ++-
datafusion/common/src/test_util.rs | 54 +-
datafusion/common/src/tree_node.rs | 54 +-
datafusion/common/src/types/builtin.rs | 15 +
datafusion/common/src/types/native.rs | 14 +-
datafusion/common/src/utils/memory.rs | 200 +-
datafusion/common/src/utils/mod.rs | 22 +-
datafusion/core/Cargo.toml | 18 +-
datafusion/core/benches/aggregate_query_sql.rs | 1 +
datafusion/core/benches/csv_load.rs | 1 +
datafusion/core/benches/data_utils/mod.rs | 3 +-
datafusion/core/benches/dataframe.rs | 1 +
datafusion/core/benches/distinct_query_sql.rs | 6 +-
datafusion/core/benches/filter_query_sql.rs | 2 +-
datafusion/core/benches/map_query_sql.rs | 4 +-
datafusion/core/benches/math_query_sql.rs | 1 +
datafusion/core/benches/parquet_query_sql.rs | 4 +-
datafusion/core/benches/physical_plan.rs | 3 +-
.../core/benches/preserve_file_partitioning.rs | 838 +++++++
datafusion/core/benches/push_down_filter.rs | 6 +-
datafusion/core/benches/scalar.rs | 2 +-
datafusion/core/benches/sort.rs | 66 +-
datafusion/core/benches/sort_limit_query_sql.rs | 4 +-
datafusion/core/benches/spm.rs | 6 +-
datafusion/core/benches/sql_planner.rs | 11 +-
datafusion/core/benches/sql_planner_extended.rs | 2 +-
datafusion/core/benches/sql_query_with_io.rs | 6 +-
datafusion/core/benches/struct_query_sql.rs | 2 +-
datafusion/core/benches/topk_aggregate.rs | 11 +-
datafusion/core/benches/window_query_sql.rs | 1 +
datafusion/core/src/bin/print_functions_docs.rs | 11 +-
datafusion/core/src/dataframe/mod.rs | 54 +-
datafusion/core/src/dataframe/parquet.rs | 2 +-
datafusion/core/src/datasource/dynamic_file.rs | 2 +-
datafusion/core/src/datasource/empty.rs | 2 +-
datafusion/core/src/datasource/file_format/avro.rs | 40 +-
datafusion/core/src/datasource/file_format/csv.rs | 159 +-
datafusion/core/src/datasource/file_format/json.rs | 68 +-
datafusion/core/src/datasource/file_format/mod.rs | 31 +-
.../core/src/datasource/file_format/options.rs | 8 +-
.../core/src/datasource/file_format/parquet.rs | 48 +-
datafusion/core/src/datasource/listing/table.rs | 425 +---
.../core/src/datasource/listing_table_factory.rs | 252 ++-
datafusion/core/src/datasource/memory_test.rs | 11 +-
datafusion/core/src/datasource/mod.rs | 124 +-
.../core/src/datasource/physical_plan/avro.rs | 96 +-
.../core/src/datasource/physical_plan/csv.rs | 182 +-
.../core/src/datasource/physical_plan/json.rs | 32 +-
.../core/src/datasource/physical_plan/mod.rs | 8 +-
.../core/src/datasource/physical_plan/parquet.rs | 324 ++-
datafusion/core/src/datasource/view_test.rs | 41 +-
datafusion/core/src/execution/context/mod.rs | 179 +-
datafusion/core/src/execution/context/parquet.rs | 6 +-
datafusion/core/src/execution/session_state.rs | 192 +-
.../core/src/execution/session_state_defaults.rs | 6 +-
datafusion/core/src/lib.rs | 8 +-
datafusion/core/src/physical_planner.rs | 434 +++-
datafusion/core/src/prelude.rs | 2 +-
datafusion/core/src/test/mod.rs | 19 +-
datafusion/core/src/test/object_store.rs | 10 +-
datafusion/core/src/test_util/mod.rs | 75 +-
datafusion/core/src/test_util/parquet.rs | 13 +-
datafusion/core/tests/catalog/memory.rs | 10 +-
.../tests/catalog_listing/pruned_partition_list.rs | 4 +-
datafusion/core/tests/config_from_env.rs | 60 +-
datafusion/core/tests/custom_sources_cases/mod.rs | 9 +-
.../provider_filter_pushdown.rs | 14 +-
.../core/tests/custom_sources_cases/statistics.rs | 2 +
.../core/tests/data/recursive_cte/closure.csv | 6 +
.../core/tests/dataframe/dataframe_functions.rs | 12 +-
datafusion/core/tests/dataframe/describe.rs | 2 +-
datafusion/core/tests/dataframe/mod.rs | 919 ++++----
.../core/tests/datasource/object_store_access.rs | 53 +
datafusion/core/tests/execution/coop.rs | 24 +-
.../core/tests/execution/datasource_split.rs | 2 +-
datafusion/core/tests/execution/logical_plan.rs | 2 +-
datafusion/core/tests/expr_api/mod.rs | 6 +-
datafusion/core/tests/expr_api/parse_sql_expr.rs | 2 +-
datafusion/core/tests/expr_api/simplification.rs | 22 +-
datafusion/core/tests/fifo/mod.rs | 8 +-
datafusion/core/tests/fuzz.rs | 5 +-
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 37 +-
.../aggregation_fuzzer/context_generator.rs | 2 +-
.../aggregation_fuzzer/data_generator.rs | 6 +-
.../tests/fuzz_cases/aggregation_fuzzer/fuzzer.rs | 4 +-
.../fuzz_cases/aggregation_fuzzer/query_builder.rs | 2 +-
.../tests/fuzz_cases/distinct_count_string_fuzz.rs | 2 +-
.../core/tests/fuzz_cases/equivalence/ordering.rs | 10 +-
.../tests/fuzz_cases/equivalence/projection.rs | 8 +-
.../tests/fuzz_cases/equivalence/properties.rs | 6 +-
.../core/tests/fuzz_cases/equivalence/utils.rs | 8 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 100 +-
datafusion/core/tests/fuzz_cases/limit_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/merge_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/mod.rs | 6 +
datafusion/core/tests/fuzz_cases/pruning.rs | 4 +-
.../tests/fuzz_cases/record_batch_generator.rs | 20 +-
datafusion/core/tests/fuzz_cases/sort_fuzz.rs | 4 +-
.../fuzz_cases/sort_preserving_repartition_fuzz.rs | 11 +-
.../core/tests/fuzz_cases/sort_query_fuzz.rs | 32 +-
.../spilling_fuzz_in_memory_constrained_env.rs | 42 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 51 +-
.../memory_limit/memory_limit_validation/utils.rs | 12 +-
datafusion/core/tests/memory_limit/mod.rs | 28 +-
.../tests/memory_limit/repartition_mem_limit.rs | 15 +-
datafusion/core/tests/optimizer/mod.rs | 6 +-
datafusion/core/tests/parquet/custom_reader.rs | 6 +-
datafusion/core/tests/parquet/encryption.rs | 5 +-
.../core/tests/parquet/external_access_plan.rs | 10 +-
datafusion/core/tests/parquet/file_statistics.rs | 28 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 23 +-
datafusion/core/tests/parquet/mod.rs | 31 +-
datafusion/core/tests/parquet/page_pruning.rs | 8 +-
datafusion/core/tests/parquet/schema_adapter.rs | 409 +---
datafusion/core/tests/parquet/schema_coercion.rs | 7 +-
datafusion/core/tests/parquet/utils.rs | 11 +-
.../physical_optimizer/aggregate_statistics.rs | 6 +-
.../combine_partial_final_agg.rs | 8 +-
.../physical_optimizer/enforce_distribution.rs | 830 ++++---
.../tests/physical_optimizer/enforce_sorting.rs | 33 +-
.../enforce_sorting_monotonicity.rs | 2 +-
.../physical_optimizer/filter_pushdown/mod.rs | 1247 +++++++++--
.../physical_optimizer/filter_pushdown/util.rs | 22 +-
.../tests/physical_optimizer/join_selection.rs | 25 +-
.../tests/physical_optimizer/limit_pushdown.rs | 148 +-
.../limited_distinct_aggregation.rs | 6 +-
datafusion/core/tests/physical_optimizer/mod.rs | 6 +
.../physical_optimizer/partition_statistics.rs | 326 ++-
.../physical_optimizer/projection_pushdown.rs | 242 +--
.../core/tests/physical_optimizer/pushdown_sort.rs | 672 ++++++
.../replace_with_order_preserving_variants.rs | 10 +-
.../tests/physical_optimizer/sanity_checker.rs | 6 +-
.../core/tests/physical_optimizer/test_utils.rs | 96 +-
.../tests/physical_optimizer/window_optimize.rs | 4 +-
.../schema_adapter_integration_tests.rs | 537 ++++-
datafusion/core/tests/sql/aggregates/basic.rs | 8 +-
datafusion/core/tests/sql/aggregates/dict_nulls.rs | 24 +-
datafusion/core/tests/sql/aggregates/mod.rs | 12 +-
datafusion/core/tests/sql/explain_analyze.rs | 59 +-
datafusion/core/tests/sql/joins.rs | 79 +-
datafusion/core/tests/sql/mod.rs | 7 +-
datafusion/core/tests/sql/path_partition.rs | 38 +-
datafusion/core/tests/sql/runtime_config.rs | 92 +
datafusion/core/tests/sql/select.rs | 10 +-
datafusion/core/tests/tpc-ds/30.sql | 4 +-
datafusion/core/tests/tracing/asserting_tracer.rs | 2 +-
.../core/tests/tracing/traceable_object_store.rs | 4 +-
datafusion/core/tests/user_defined/expr_planner.rs | 18 +-
.../core/tests/user_defined/insert_operation.rs | 4 +-
datafusion/core/tests/user_defined/mod.rs | 3 +
.../core/tests/user_defined/relation_planner.rs | 527 +++++
.../tests/user_defined/user_defined_aggregates.rs | 81 +-
.../user_defined_async_scalar_functions.rs | 2 +-
.../core/tests/user_defined/user_defined_plan.rs | 73 +-
.../user_defined/user_defined_scalar_functions.rs | 69 +-
.../user_defined/user_defined_table_functions.rs | 16 +-
.../user_defined/user_defined_window_functions.rs | 303 +--
datafusion/datasource-arrow/src/file_format.rs | 34 +-
datafusion/datasource-arrow/src/mod.rs | 3 +-
datafusion/datasource-arrow/src/source.rs | 195 +-
.../src/avro_to_arrow/arrow_array_reader.rs | 58 +-
.../datasource-avro/src/avro_to_arrow/schema.rs | 16 +-
datafusion/datasource-avro/src/file_format.rs | 12 +-
datafusion/datasource-avro/src/mod.rs | 2 +-
datafusion/datasource-avro/src/source.rs | 53 +-
datafusion/datasource-csv/Cargo.toml | 2 +-
datafusion/datasource-csv/src/file_format.rs | 17 +-
datafusion/datasource-csv/src/mod.rs | 14 +-
datafusion/datasource-csv/src/source.rs | 95 +-
datafusion/datasource-json/src/file_format.rs | 17 +-
datafusion/datasource-json/src/mod.rs | 3 +-
datafusion/datasource-json/src/source.rs | 51 +-
datafusion/datasource-parquet/src/access_plan.rs | 18 +-
datafusion/datasource-parquet/src/file_format.rs | 37 +-
datafusion/datasource-parquet/src/metadata.rs | 209 +-
datafusion/datasource-parquet/src/mod.rs | 4 +-
datafusion/datasource-parquet/src/opener.rs | 1038 ++++++---
datafusion/datasource-parquet/src/page_filter.rs | 18 +-
datafusion/datasource-parquet/src/reader.rs | 7 +-
datafusion/datasource-parquet/src/row_filter.rs | 287 ++-
.../datasource-parquet/src/row_group_filter.rs | 11 +-
datafusion/datasource-parquet/src/sort.rs | 407 ++++
datafusion/datasource-parquet/src/source.rs | 326 ++-
datafusion/datasource/Cargo.toml | 12 +-
.../benches/split_groups_by_statistics.rs | 2 +-
datafusion/datasource/src/decoder.rs | 28 +-
datafusion/datasource/src/display.rs | 2 +-
datafusion/datasource/src/file.rs | 95 +-
datafusion/datasource/src/file_compression_type.rs | 60 +-
datafusion/datasource/src/file_format.rs | 2 +-
datafusion/datasource/src/file_groups.rs | 379 +++-
datafusion/datasource/src/file_scan_config.rs | 1446 +++++--------
datafusion/datasource/src/file_sink_config.rs | 4 +-
datafusion/datasource/src/file_stream.rs | 143 +-
datafusion/datasource/src/memory.rs | 33 +-
datafusion/datasource/src/mod.rs | 153 +-
datafusion/datasource/src/projection.rs | 630 ++++++
datafusion/datasource/src/schema_adapter.rs | 15 +-
datafusion/datasource/src/sink.rs | 6 +-
datafusion/datasource/src/source.rs | 52 +-
datafusion/datasource/src/statistics.rs | 12 +-
datafusion/datasource/src/table_schema.rs | 107 +-
datafusion/datasource/src/test_util.rs | 41 +-
datafusion/datasource/src/url.rs | 287 ++-
datafusion/datasource/src/write/demux.rs | 16 +-
datafusion/datasource/src/write/mod.rs | 25 +-
datafusion/datasource/src/write/orchestration.rs | 24 +-
datafusion/doc/src/lib.rs | 11 +-
datafusion/doc/src/udf.rs | 4 +-
datafusion/execution/Cargo.toml | 2 +-
datafusion/execution/src/cache/cache_manager.rs | 152 +-
datafusion/execution/src/cache/cache_unit.rs | 842 +-------
.../{cache_unit.rs => file_metadata_cache.rs} | 251 +--
datafusion/execution/src/cache/list_files_cache.rs | 1072 +++++++++
datafusion/execution/src/cache/mod.rs | 15 +-
datafusion/execution/src/config.rs | 9 +-
datafusion/execution/src/disk_manager.rs | 20 +-
datafusion/execution/src/lib.rs | 6 +-
datafusion/execution/src/memory_pool/mod.rs | 138 +-
datafusion/execution/src/memory_pool/pool.rs | 57 +-
.../src/metrics/baseline.rs | 8 +-
.../src/metrics/builder.rs | 4 +-
.../src/metrics/custom.rs | 2 +-
.../src/metrics/mod.rs | 32 +-
.../src/metrics/value.rs | 16 +-
datafusion/execution/src/object_store.rs | 30 +-
datafusion/execution/src/parquet_encryption.rs | 2 +-
datafusion/execution/src/runtime_env.rs | 68 +-
datafusion/execution/src/task.rs | 7 +-
datafusion/expr-common/Cargo.toml | 2 +-
datafusion/expr-common/src/accumulator.rs | 2 +-
datafusion/expr-common/src/casts.rs | 9 +-
datafusion/expr-common/src/columnar_value.rs | 4 +-
datafusion/expr-common/src/dyn_eq.rs | 4 +-
datafusion/expr-common/src/groups_accumulator.rs | 2 +-
datafusion/expr-common/src/interval_arithmetic.rs | 58 +-
datafusion/expr-common/src/lib.rs | 3 +-
datafusion/expr-common/src/signature.rs | 190 +-
datafusion/expr-common/src/statistics.rs | 10 +-
.../expr-common/src/type_coercion/aggregates.rs | 2 +-
datafusion/expr-common/src/type_coercion/binary.rs | 24 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 13 +-
.../src/type_coercion/binary/tests/comparison.rs | 10 +-
datafusion/expr/Cargo.toml | 2 +-
datafusion/expr/src/arguments.rs | 42 +-
datafusion/expr/src/async_udf.rs | 2 +-
datafusion/expr/src/conditional_expressions.rs | 4 +-
datafusion/expr/src/execution_props.rs | 7 +-
datafusion/expr/src/expr.rs | 61 +-
datafusion/expr/src/expr_fn.rs | 4 +-
datafusion/expr/src/expr_rewriter/guarantees.rs | 14 +-
datafusion/expr/src/expr_rewriter/mod.rs | 14 +-
datafusion/expr/src/expr_rewriter/order_by.rs | 6 +-
datafusion/expr/src/expr_schema.rs | 170 +-
datafusion/expr/src/lib.rs | 17 +-
datafusion/expr/src/literal.rs | 2 +-
datafusion/expr/src/logical_plan/builder.rs | 172 +-
datafusion/expr/src/logical_plan/ddl.rs | 159 +-
datafusion/expr/src/logical_plan/display.rs | 43 +-
datafusion/expr/src/logical_plan/invariants.rs | 16 +-
datafusion/expr/src/logical_plan/mod.rs | 15 +-
datafusion/expr/src/logical_plan/plan.rs | 186 +-
datafusion/expr/src/logical_plan/statement.rs | 2 +-
datafusion/expr/src/logical_plan/tree_node.rs | 12 +-
datafusion/expr/src/partition_evaluator.rs | 2 +-
datafusion/expr/src/planner.rs | 107 +-
datafusion/expr/src/predicate_bounds.rs | 12 +-
datafusion/expr/src/ptr_eq.rs | 2 +-
datafusion/expr/src/registry.rs | 2 +-
datafusion/expr/src/select_expr.rs | 2 +-
datafusion/expr/src/simplify.rs | 14 +-
datafusion/expr/src/test/function_stub.rs | 12 +-
datafusion/expr/src/tree_node.rs | 4 +-
datafusion/expr/src/type_coercion/functions.rs | 96 +-
datafusion/expr/src/udaf.rs | 27 +-
datafusion/expr/src/udf.rs | 18 +-
datafusion/expr/src/udf_eq.rs | 2 +-
datafusion/expr/src/udwf.rs | 12 +-
datafusion/expr/src/utils.rs | 67 +-
datafusion/expr/src/window_frame.rs | 54 +-
datafusion/expr/src/window_state.rs | 17 +-
datafusion/ffi/Cargo.toml | 13 +
datafusion/ffi/README.md | 92 +
datafusion/ffi/src/arrow_wrappers.rs | 31 +-
datafusion/ffi/src/catalog_provider.rs | 209 +-
datafusion/ffi/src/catalog_provider_list.rs | 155 +-
.../equivalence => ffi/src/execution}/mod.rs | 9 +-
datafusion/ffi/src/execution/task_ctx.rs | 288 +++
datafusion/ffi/src/execution/task_ctx_provider.rs | 229 ++
datafusion/ffi/src/execution_plan.rs | 189 +-
datafusion/ffi/src/expr/columnar_value.rs | 85 +
datafusion/ffi/src/expr/distribution.rs | 215 ++
datafusion/ffi/src/expr/expr_properties.rs | 116 +
datafusion/ffi/src/expr/interval.rs | 58 +
.../{core/tests/catalog => ffi/src/expr}/mod.rs | 5 +-
datafusion/ffi/src/lib.rs | 37 +-
datafusion/ffi/src/physical_expr/mod.rs | 995 +++++++++
datafusion/ffi/src/physical_expr/partitioning.rs | 98 +
datafusion/ffi/src/physical_expr/sort.rs | 75 +
datafusion/ffi/src/plan_properties.rs | 106 +-
.../ffi/src/proto/logical_extension_codec.rs | 710 ++++++
.../{core/tests/catalog => ffi/src/proto}/mod.rs | 3 +-
.../ffi/src/proto/physical_extension_codec.rs | 682 ++++++
datafusion/ffi/src/record_batch_stream.rs | 72 +-
datafusion/ffi/src/schema_provider.rs | 225 +-
.../src/{session_config.rs => session/config.rs} | 106 +-
datafusion/ffi/src/session/mod.rs | 623 ++++++
datafusion/ffi/src/table_provider.rs | 216 +-
datafusion/ffi/src/tests/catalog.rs | 8 +-
datafusion/ffi/src/tests/mod.rs | 3 +-
datafusion/ffi/src/udaf/accumulator.rs | 211 +-
datafusion/ffi/src/udaf/accumulator_args.rs | 8 +-
datafusion/ffi/src/udaf/groups_accumulator.rs | 223 +-
datafusion/ffi/src/udaf/mod.rs | 350 +--
datafusion/ffi/src/udf/mod.rs | 235 +-
datafusion/ffi/src/udf/return_type_args.rs | 6 +-
datafusion/ffi/src/udtf.rs | 75 +-
datafusion/ffi/src/udwf/mod.rs | 294 +--
datafusion/ffi/src/udwf/partition_evaluator.rs | 231 +-
.../ffi/src/udwf/partition_evaluator_args.rs | 12 +-
datafusion/ffi/src/util.rs | 59 +-
datafusion/ffi/tests/ffi_catalog.rs | 12 +-
datafusion/ffi/tests/ffi_integration.rs | 8 +-
datafusion/ffi/tests/ffi_udaf.rs | 14 +-
datafusion/ffi/tests/ffi_udf.rs | 15 +-
datafusion/ffi/tests/ffi_udtf.rs | 10 +-
datafusion/ffi/tests/ffi_udwf.rs | 10 +-
.../benches/accumulate.rs | 2 +-
.../src/aggregate/avg_distinct/decimal.rs | 4 +-
.../src/aggregate/count_distinct/bytes.rs | 10 +-
.../src/aggregate/count_distinct/dict.rs | 4 +-
.../src/aggregate/count_distinct/native.rs | 10 +-
.../src/aggregate/groups_accumulator.rs | 2 +-
.../src/aggregate/groups_accumulator/accumulate.rs | 64 +-
.../src/aggregate/groups_accumulator/nulls.rs | 2 +-
.../src/aggregate/groups_accumulator/prim_op.rs | 2 +-
datafusion/functions-aggregate-common/src/lib.rs | 4 +-
.../functions-aggregate-common/src/merge_arrays.rs | 2 +-
.../functions-aggregate-common/src/min_max.rs | 12 +-
.../functions-aggregate-common/src/tdigest.rs | 62 +-
datafusion/functions-aggregate-common/src/utils.rs | 8 +-
datafusion/functions-aggregate/Cargo.toml | 2 +-
.../functions-aggregate/benches/array_agg.rs | 7 +-
datafusion/functions-aggregate/benches/count.rs | 3 +-
.../functions-aggregate/benches/min_max_bytes.rs | 4 +-
datafusion/functions-aggregate/benches/sum.rs | 5 +-
.../functions-aggregate/src/approx_distinct.rs | 44 +-
.../functions-aggregate/src/approx_median.rs | 90 +-
.../src/approx_percentile_cont.rs | 104 +-
.../src/approx_percentile_cont_with_weight.rs | 9 +-
datafusion/functions-aggregate/src/array_agg.rs | 40 +-
datafusion/functions-aggregate/src/average.rs | 30 +-
.../functions-aggregate/src/bit_and_or_xor.rs | 58 +-
datafusion/functions-aggregate/src/bool_and_or.rs | 32 +-
datafusion/functions-aggregate/src/correlation.rs | 28 +-
datafusion/functions-aggregate/src/count.rs | 99 +-
datafusion/functions-aggregate/src/covariance.rs | 5 +-
datafusion/functions-aggregate/src/first_last.rs | 137 +-
datafusion/functions-aggregate/src/grouping.rs | 16 +-
datafusion/functions-aggregate/src/lib.rs | 4 +-
datafusion/functions-aggregate/src/median.rs | 80 +-
datafusion/functions-aggregate/src/min_max.rs | 28 +-
.../src/min_max/min_max_bytes.rs | 2 +-
.../src/min_max/min_max_struct.rs | 3 +-
datafusion/functions-aggregate/src/nth_value.rs | 12 +-
.../functions-aggregate/src/percentile_cont.rs | 118 +-
datafusion/functions-aggregate/src/planner.rs | 2 +-
datafusion/functions-aggregate/src/regr.rs | 4 +-
datafusion/functions-aggregate/src/stddev.rs | 20 +-
datafusion/functions-aggregate/src/string_agg.rs | 23 +-
datafusion/functions-aggregate/src/sum.rs | 48 +-
datafusion/functions-aggregate/src/utils.rs | 4 +-
datafusion/functions-aggregate/src/variance.rs | 6 +-
datafusion/functions-nested/Cargo.toml | 10 +-
datafusion/functions-nested/benches/array_has.rs | 377 ++++
datafusion/functions-nested/benches/array_slice.rs | 230 ++
datafusion/functions-nested/benches/map.rs | 6 +-
datafusion/functions-nested/src/array_has.rs | 26 +-
datafusion/functions-nested/src/cardinality.rs | 7 +-
datafusion/functions-nested/src/concat.rs | 15 +-
datafusion/functions-nested/src/dimension.rs | 8 +-
datafusion/functions-nested/src/distance.rs | 6 +-
datafusion/functions-nested/src/empty.rs | 5 +-
datafusion/functions-nested/src/except.rs | 9 +-
datafusion/functions-nested/src/extract.rs | 14 +-
datafusion/functions-nested/src/flatten.rs | 5 +-
datafusion/functions-nested/src/length.rs | 5 +-
datafusion/functions-nested/src/lib.rs | 4 +-
datafusion/functions-nested/src/make_array.rs | 45 +-
datafusion/functions-nested/src/map.rs | 2 +-
datafusion/functions-nested/src/map_entries.rs | 2 +-
datafusion/functions-nested/src/map_extract.rs | 4 +-
datafusion/functions-nested/src/map_keys.rs | 2 +-
datafusion/functions-nested/src/map_values.rs | 2 +-
datafusion/functions-nested/src/min_max.rs | 15 +-
datafusion/functions-nested/src/planner.rs | 6 +-
datafusion/functions-nested/src/position.rs | 13 +-
datafusion/functions-nested/src/range.rs | 16 +-
datafusion/functions-nested/src/remove.rs | 491 ++++-
datafusion/functions-nested/src/repeat.rs | 9 +-
datafusion/functions-nested/src/replace.rs | 12 +-
datafusion/functions-nested/src/resize.rs | 11 +-
datafusion/functions-nested/src/reverse.rs | 2 +-
datafusion/functions-nested/src/set_ops.rs | 17 +-
datafusion/functions-nested/src/sort.rs | 10 +-
datafusion/functions-nested/src/string.rs | 79 +-
datafusion/functions-nested/src/utils.rs | 2 +-
datafusion/functions-table/Cargo.toml | 2 +-
datafusion/functions-table/src/generate_series.rs | 40 +-
datafusion/functions-table/src/lib.rs | 4 +-
datafusion/functions-window-common/src/lib.rs | 4 +-
datafusion/functions-window/Cargo.toml | 2 +-
datafusion/functions-window/src/cume_dist.rs | 4 +-
datafusion/functions-window/src/lead_lag.rs | 4 +-
datafusion/functions-window/src/lib.rs | 6 +-
datafusion/functions-window/src/nth_value.rs | 41 +-
datafusion/functions-window/src/ntile.rs | 2 +-
datafusion/functions-window/src/planner.rs | 2 +-
datafusion/functions-window/src/rank.rs | 7 +-
datafusion/functions-window/src/utils.rs | 2 +-
datafusion/functions/Cargo.toml | 4 +-
datafusion/functions/benches/ascii.rs | 2 +-
datafusion/functions/benches/character_length.rs | 2 +-
datafusion/functions/benches/chr.rs | 2 +-
datafusion/functions/benches/concat.rs | 4 +-
datafusion/functions/benches/cot.rs | 2 +-
datafusion/functions/benches/date_bin.rs | 6 +-
datafusion/functions/benches/date_trunc.rs | 6 +-
datafusion/functions/benches/encoding.rs | 2 +-
datafusion/functions/benches/find_in_set.rs | 4 +-
datafusion/functions/benches/gcd.rs | 4 +-
datafusion/functions/benches/helper.rs | 2 +-
datafusion/functions/benches/initcap.rs | 2 +-
datafusion/functions/benches/isnan.rs | 2 +-
datafusion/functions/benches/iszero.rs | 2 +-
datafusion/functions/benches/lower.rs | 2 +-
datafusion/functions/benches/ltrim.rs | 8 +-
datafusion/functions/benches/make_date.rs | 6 +-
datafusion/functions/benches/nullif.rs | 4 +-
datafusion/functions/benches/pad.rs | 7 +-
datafusion/functions/benches/random.rs | 2 +-
datafusion/functions/benches/regx.rs | 4 +-
datafusion/functions/benches/repeat.rs | 4 +-
datafusion/functions/benches/reverse.rs | 2 +-
datafusion/functions/benches/signum.rs | 2 +-
datafusion/functions/benches/strpos.rs | 2 +-
datafusion/functions/benches/substr.rs | 5 +-
datafusion/functions/benches/substr_index.rs | 4 +-
datafusion/functions/benches/to_char.rs | 8 +-
datafusion/functions/benches/to_hex.rs | 2 +-
datafusion/functions/benches/to_timestamp.rs | 2 +-
datafusion/functions/benches/trunc.rs | 2 +-
datafusion/functions/benches/upper.rs | 2 +-
datafusion/functions/benches/uuid.rs | 2 +-
datafusion/functions/src/core/arrow_cast.rs | 6 +-
datafusion/functions/src/core/arrowtypeof.rs | 2 +-
datafusion/functions/src/core/coalesce.rs | 2 +-
datafusion/functions/src/core/getfield.rs | 74 +-
datafusion/functions/src/core/greatest.rs | 14 +-
.../functions/src/core/greatest_least_utils.rs | 4 +-
datafusion/functions/src/core/least.rs | 14 +-
datafusion/functions/src/core/named_struct.rs | 2 +-
datafusion/functions/src/core/nullif.rs | 2 +-
datafusion/functions/src/core/nvl2.rs | 6 +-
datafusion/functions/src/core/overlay.rs | 6 +-
datafusion/functions/src/core/planner.rs | 2 +-
datafusion/functions/src/core/struct.rs | 2 +-
datafusion/functions/src/core/union_extract.rs | 15 +-
datafusion/functions/src/core/union_tag.rs | 4 +-
datafusion/functions/src/core/version.rs | 4 +-
datafusion/functions/src/crypto/basic.rs | 163 +-
datafusion/functions/src/crypto/digest.rs | 48 +-
datafusion/functions/src/crypto/md5.rs | 52 +-
datafusion/functions/src/crypto/mod.rs | 13 +-
datafusion/functions/src/crypto/sha.rs | 175 ++
datafusion/functions/src/crypto/sha224.rs | 104 -
datafusion/functions/src/crypto/sha256.rs | 102 -
datafusion/functions/src/crypto/sha384.rs | 102 -
datafusion/functions/src/crypto/sha512.rs | 102 -
datafusion/functions/src/datetime/common.rs | 26 +-
datafusion/functions/src/datetime/current_date.rs | 2 +-
datafusion/functions/src/datetime/current_time.rs | 7 +-
datafusion/functions/src/datetime/date_bin.rs | 16 +-
datafusion/functions/src/datetime/date_part.rs | 6 +-
datafusion/functions/src/datetime/date_trunc.rs | 14 +-
datafusion/functions/src/datetime/from_unixtime.rs | 4 +-
datafusion/functions/src/datetime/make_date.rs | 337 +--
datafusion/functions/src/datetime/make_time.rs | 267 +++
datafusion/functions/src/datetime/mod.rs | 7 +
datafusion/functions/src/datetime/now.rs | 4 +-
datafusion/functions/src/datetime/planner.rs | 2 +-
datafusion/functions/src/datetime/to_char.rs | 10 +-
datafusion/functions/src/datetime/to_date.rs | 53 +-
datafusion/functions/src/datetime/to_local_time.rs | 347 ++-
datafusion/functions/src/datetime/to_timestamp.rs | 31 +-
datafusion/functions/src/datetime/to_unixtime.rs | 2 +-
datafusion/functions/src/encoding/inner.rs | 31 +-
datafusion/functions/src/lib.rs | 4 +-
datafusion/functions/src/macros.rs | 16 +-
datafusion/functions/src/math/abs.rs | 8 +-
datafusion/functions/src/math/cot.rs | 2 +-
datafusion/functions/src/math/factorial.rs | 2 +-
datafusion/functions/src/math/gcd.rs | 35 +-
datafusion/functions/src/math/iszero.rs | 4 +-
datafusion/functions/src/math/lcm.rs | 2 +-
datafusion/functions/src/math/log.rs | 17 +-
datafusion/functions/src/math/monotonicity.rs | 4 +-
datafusion/functions/src/math/nans.rs | 4 +-
datafusion/functions/src/math/nanvl.rs | 14 +-
datafusion/functions/src/math/pi.rs | 2 +-
datafusion/functions/src/math/power.rs | 497 ++---
datafusion/functions/src/math/random.rs | 4 +-
datafusion/functions/src/math/round.rs | 10 +-
datafusion/functions/src/math/signum.rs | 16 +-
datafusion/functions/src/math/trunc.rs | 16 +-
datafusion/functions/src/planner.rs | 2 +-
datafusion/functions/src/regex/mod.rs | 2 +-
datafusion/functions/src/regex/regexpcount.rs | 16 +-
datafusion/functions/src/regex/regexpinstr.rs | 56 +-
datafusion/functions/src/regex/regexplike.rs | 51 +-
datafusion/functions/src/regex/regexpmatch.rs | 33 +-
datafusion/functions/src/regex/regexpreplace.rs | 69 +-
datafusion/functions/src/string/ascii.rs | 2 +-
datafusion/functions/src/string/btrim.rs | 2 +-
datafusion/functions/src/string/chr.rs | 14 +-
datafusion/functions/src/string/common.rs | 14 +-
datafusion/functions/src/string/concat.rs | 37 +-
datafusion/functions/src/string/concat_ws.rs | 64 +-
datafusion/functions/src/string/contains.rs | 4 +-
datafusion/functions/src/string/ends_with.rs | 8 +-
datafusion/functions/src/string/levenshtein.rs | 10 +-
datafusion/functions/src/string/lower.rs | 2 +-
datafusion/functions/src/string/ltrim.rs | 2 +-
datafusion/functions/src/string/octet_length.rs | 2 +-
datafusion/functions/src/string/repeat.rs | 6 +-
datafusion/functions/src/string/replace.rs | 6 +-
datafusion/functions/src/string/rtrim.rs | 2 +-
datafusion/functions/src/string/split_part.rs | 50 +-
datafusion/functions/src/string/starts_with.rs | 27 +-
datafusion/functions/src/string/to_hex.rs | 14 +-
datafusion/functions/src/string/upper.rs | 2 +-
datafusion/functions/src/string/uuid.rs | 4 +-
datafusion/functions/src/strings.rs | 4 +-
.../functions/src/unicode/character_length.rs | 4 +-
datafusion/functions/src/unicode/find_in_set.rs | 19 +-
datafusion/functions/src/unicode/initcap.rs | 2 +-
datafusion/functions/src/unicode/left.rs | 4 +-
datafusion/functions/src/unicode/lpad.rs | 14 +-
datafusion/functions/src/unicode/planner.rs | 2 +-
datafusion/functions/src/unicode/reverse.rs | 6 +-
datafusion/functions/src/unicode/right.rs | 6 +-
datafusion/functions/src/unicode/rpad.rs | 10 +-
datafusion/functions/src/unicode/strpos.rs | 2 +-
datafusion/functions/src/unicode/substr.rs | 10 +-
datafusion/functions/src/unicode/substrindex.rs | 4 +-
datafusion/functions/src/unicode/translate.rs | 2 +-
datafusion/functions/src/utils.rs | 5 +-
datafusion/macros/Cargo.toml | 2 +-
datafusion/macros/src/user_doc.rs | 3 +-
.../optimizer/benches/projection_unnecessary.rs | 4 +-
.../optimizer/src/analyzer/function_rewrite.rs | 2 +-
datafusion/optimizer/src/analyzer/mod.rs | 2 +-
.../src/analyzer/resolve_grouping_function.rs | 8 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 51 +-
.../optimizer/src/common_subexpr_eliminate.rs | 14 +-
datafusion/optimizer/src/decorrelate.rs | 6 +-
.../optimizer/src/decorrelate_lateral_join.rs | 6 +-
.../src/decorrelate_predicate_subquery.rs | 18 +-
datafusion/optimizer/src/eliminate_cross_join.rs | 19 +-
.../optimizer/src/eliminate_duplicated_expr.rs | 6 +-
datafusion/optimizer/src/eliminate_filter.rs | 6 +-
.../optimizer/src/eliminate_group_by_constant.rs | 8 +-
datafusion/optimizer/src/eliminate_join.rs | 4 +-
datafusion/optimizer/src/eliminate_limit.rs | 10 +-
datafusion/optimizer/src/eliminate_outer_join.rs | 6 +-
.../optimizer/src/extract_equijoin_predicate.rs | 6 +-
datafusion/optimizer/src/filter_null_join_keys.rs | 6 +-
datafusion/optimizer/src/join_key_set.rs | 2 +-
datafusion/optimizer/src/lib.rs | 3 +-
.../optimizer/src/optimize_projections/mod.rs | 31 +-
datafusion/optimizer/src/optimize_unions.rs | 67 +-
datafusion/optimizer/src/optimizer.rs | 16 +-
datafusion/optimizer/src/plan_signature.rs | 2 +-
.../optimizer/src/propagate_empty_relation.rs | 18 +-
datafusion/optimizer/src/push_down_filter.rs | 24 +-
datafusion/optimizer/src/push_down_limit.rs | 16 +-
.../optimizer/src/replace_distinct_aggregate.rs | 6 +-
.../optimizer/src/scalar_subquery_to_join.rs | 52 +-
.../src/simplify_expressions/expr_simplifier.rs | 38 +-
.../src/simplify_expressions/inlist_simplifier.rs | 92 +-
.../optimizer/src/simplify_expressions/regex.rs | 20 +-
.../src/simplify_expressions/simplify_exprs.rs | 27 +-
.../src/simplify_expressions/unwrap_cast.rs | 6 +-
.../optimizer/src/simplify_expressions/utils.rs | 4 +-
.../optimizer/src/single_distinct_to_groupby.rs | 9 +-
datafusion/optimizer/src/test/mod.rs | 4 +-
datafusion/optimizer/src/test/user_defined.rs | 2 +-
datafusion/optimizer/src/utils.rs | 6 +-
.../optimizer/tests/optimizer_integration.rs | 369 ++--
datafusion/physical-expr-adapter/src/lib.rs | 6 +-
.../physical-expr-adapter/src/schema_rewriter.rs | 449 ++--
datafusion/physical-expr-common/src/binary_map.rs | 4 +-
datafusion/physical-expr-common/src/datum.rs | 7 +-
datafusion/physical-expr-common/src/lib.rs | 4 +-
.../physical-expr-common/src/physical_expr.rs | 56 +-
datafusion/physical-expr-common/src/sort_expr.rs | 105 +-
datafusion/physical-expr-common/src/tree_node.rs | 4 +-
datafusion/physical-expr-common/src/utils.rs | 4 +-
datafusion/physical-expr/Cargo.toml | 4 +-
datafusion/physical-expr/benches/binary_op.rs | 7 +-
datafusion/physical-expr/benches/case_when.rs | 12 +-
datafusion/physical-expr/benches/in_list.rs | 205 +-
datafusion/physical-expr/benches/is_null.rs | 4 +-
datafusion/physical-expr/src/aggregate.rs | 37 +-
datafusion/physical-expr/src/analysis.rs | 52 +-
.../physical-expr/src/async_scalar_function.rs | 13 +-
datafusion/physical-expr/src/equivalence/class.rs | 35 +-
datafusion/physical-expr/src/equivalence/mod.rs | 6 +-
.../physical-expr/src/equivalence/ordering.rs | 15 +-
.../src/equivalence/properties/dependency.rs | 6 +-
.../src/equivalence/properties/joins.rs | 2 +-
.../src/equivalence/properties/mod.rs | 8 +-
.../src/equivalence/properties/union.rs | 4 +-
datafusion/physical-expr/src/expressions/binary.rs | 98 +-
.../src/expressions/binary/kernels.rs | 43 +-
datafusion/physical-expr/src/expressions/case.rs | 732 ++++++-
.../literal_lookup_table/boolean_lookup_table.rs | 122 ++
.../bytes_like_lookup_table.rs | 223 ++
.../expressions/case/literal_lookup_table/mod.rs | 327 +++
.../literal_lookup_table/primitive_lookup_table.rs | 229 ++
datafusion/physical-expr/src/expressions/cast.rs | 34 +-
.../physical-expr/src/expressions/cast_column.rs | 4 +-
datafusion/physical-expr/src/expressions/column.rs | 8 +-
.../src/expressions/dynamic_filters.rs | 88 +-
.../physical-expr/src/expressions/in_list.rs | 1788 +++++++++------
datafusion/physical-expr/src/expressions/like.rs | 2 +-
datafusion/physical-expr/src/expressions/mod.rs | 26 +-
.../physical-expr/src/expressions/negative.rs | 20 +-
datafusion/physical-expr/src/expressions/no_op.rs | 2 +-
datafusion/physical-expr/src/expressions/not.rs | 41 +-
.../physical-expr/src/expressions/try_cast.rs | 6 +-
.../src/expressions/unknown_column.rs | 2 +-
.../physical-expr/src/intervals/cp_solver.rs | 12 +-
.../physical-expr/src/intervals/test_utils.rs | 6 +-
datafusion/physical-expr/src/intervals/utils.rs | 6 +-
datafusion/physical-expr/src/lib.rs | 8 +-
datafusion/physical-expr/src/partitioning.rs | 4 +-
datafusion/physical-expr/src/physical_expr.rs | 4 +-
datafusion/physical-expr/src/planner.rs | 8 +-
datafusion/physical-expr/src/projection.rs | 228 +-
datafusion/physical-expr/src/scalar_function.rs | 37 +-
.../src/simplifier/const_evaluator.rs | 103 +
datafusion/physical-expr/src/simplifier/mod.rs | 538 ++++-
datafusion/physical-expr/src/simplifier/not.rs | 124 ++
.../physical-expr/src/simplifier/unwrap_cast.rs | 53 +-
.../physical-expr/src/statistics/stats_solver.rs | 6 +-
datafusion/physical-expr/src/utils/guarantee.rs | 4 +-
datafusion/physical-expr/src/utils/mod.rs | 8 +-
datafusion/physical-expr/src/window/aggregate.rs | 4 +-
.../physical-expr/src/window/sliding_aggregate.rs | 4 +-
datafusion/physical-expr/src/window/standard.rs | 6 +-
.../src/window/standard_window_function_expr.rs | 3 +-
datafusion/physical-expr/src/window/window_expr.rs | 9 +-
.../physical-optimizer/src/aggregate_statistics.rs | 45 +-
.../physical-optimizer/src/coalesce_batches.rs | 30 +-
.../src/combine_partial_final_agg.rs | 6 +-
.../physical-optimizer/src/enforce_distribution.rs | 248 +--
.../physical-optimizer/src/enforce_sorting/mod.rs | 12 +-
.../replace_with_order_preserving_variants.rs | 4 +-
.../src/enforce_sorting/sort_pushdown.rs | 6 +-
datafusion/physical-optimizer/src/ensure_coop.rs | 12 +-
.../physical-optimizer/src/filter_pushdown.rs | 8 +-
.../physical-optimizer/src/join_selection.rs | 28 +-
datafusion/physical-optimizer/src/lib.rs | 5 +-
.../physical-optimizer/src/limit_pushdown.rs | 2 +-
.../src/limit_pushdown_past_window.rs | 10 +-
.../src/limited_distinct_aggregation.rs | 20 +-
datafusion/physical-optimizer/src/optimizer.rs | 5 +-
.../physical-optimizer/src/output_requirements.rs | 2 +-
.../physical-optimizer/src/projection_pushdown.rs | 10 +-
datafusion/physical-optimizer/src/pushdown_sort.rs | 129 ++
.../physical-optimizer/src/sanity_checker.rs | 17 +-
.../physical-optimizer/src/topk_aggregation.rs | 4 +-
.../physical-optimizer/src/update_aggr_exprs.rs | 6 +-
datafusion/physical-plan/Cargo.toml | 4 +-
.../physical-plan/benches/aggregate_vectorized.rs | 5 +-
.../physical-plan/benches/partial_ordering.rs | 2 +-
.../physical-plan/benches/sort_preserving_merge.rs | 4 +-
datafusion/physical-plan/benches/spill_io.rs | 7 +-
.../src/aggregates/group_values/metrics.rs | 2 +-
.../src/aggregates/group_values/mod.rs | 2 +-
.../group_values/multi_group_by/boolean.rs | 2 +-
.../group_values/multi_group_by/bytes.rs | 10 +-
.../group_values/multi_group_by/bytes_view.rs | 4 +-
.../aggregates/group_values/multi_group_by/mod.rs | 14 +-
.../group_values/multi_group_by/primitive.rs | 4 +-
.../src/aggregates/group_values/row.rs | 2 +-
.../group_values/single_group_by/primitive.rs | 6 +-
datafusion/physical-plan/src/aggregates/mod.rs | 744 +++++--
.../physical-plan/src/aggregates/no_grouping.rs | 263 ++-
.../physical-plan/src/aggregates/row_hash.rs | 185 +-
.../src/aggregates/topk/hash_table.rs | 224 +-
.../physical-plan/src/aggregates/topk/heap.rs | 113 +-
.../src/aggregates/topk/priority_map.rs | 146 +-
.../physical-plan/src/aggregates/topk_stream.rs | 8 +-
datafusion/physical-plan/src/analyze.rs | 4 +-
datafusion/physical-plan/src/async_func.rs | 27 +-
datafusion/physical-plan/src/coalesce/mod.rs | 2 +-
datafusion/physical-plan/src/coalesce_batches.rs | 16 +
.../physical-plan/src/coalesce_partitions.rs | 46 +-
datafusion/physical-plan/src/common.rs | 5 +-
datafusion/physical-plan/src/coop.rs | 4 +-
datafusion/physical-plan/src/display.rs | 4 +-
datafusion/physical-plan/src/empty.rs | 13 +-
datafusion/physical-plan/src/execution_plan.rs | 42 +-
datafusion/physical-plan/src/explain.rs | 15 +-
datafusion/physical-plan/src/filter.rs | 66 +-
datafusion/physical-plan/src/joins/cross_join.rs | 65 +-
.../physical-plan/src/joins/hash_join/exec.rs | 357 +--
.../src/joins/hash_join/inlist_builder.rs | 133 ++
.../physical-plan/src/joins/hash_join/mod.rs | 2 +
.../src/joins/hash_join/partitioned_hash_eval.rs | 146 +-
.../src/joins/hash_join/shared_bounds.rs | 401 +++-
.../physical-plan/src/joins/hash_join/stream.rs | 210 +-
.../physical-plan/src/joins/join_hash_map.rs | 165 +-
datafusion/physical-plan/src/joins/mod.rs | 8 +-
.../physical-plan/src/joins/nested_loop_join.rs | 299 +--
.../src/joins/piecewise_merge_join/classic_join.rs | 147 +-
.../src/joins/piecewise_merge_join/exec.rs | 23 +-
.../src/joins/sort_merge_join/exec.rs | 20 +-
.../src/joins/sort_merge_join/metrics.rs | 2 -
.../src/joins/sort_merge_join/stream.rs | 713 +++---
.../src/joins/sort_merge_join/tests.rs | 823 +++----
.../physical-plan/src/joins/stream_join_utils.rs | 118 +-
.../physical-plan/src/joins/symmetric_hash_join.rs | 34 +-
datafusion/physical-plan/src/joins/test_utils.rs | 6 +-
datafusion/physical-plan/src/joins/utils.rs | 106 +-
datafusion/physical-plan/src/lib.rs | 20 +-
datafusion/physical-plan/src/limit.rs | 13 +-
datafusion/physical-plan/src/memory.rs | 6 +-
.../src/metrics.rs} | 8 +-
datafusion/physical-plan/src/placeholder_row.rs | 13 +-
datafusion/physical-plan/src/projection.rs | 29 +-
datafusion/physical-plan/src/recursive_query.rs | 108 +-
datafusion/physical-plan/src/render_tree.rs | 3 +-
.../src/repartition/distributor_channels.rs | 4 +-
datafusion/physical-plan/src/repartition/mod.rs | 223 +-
datafusion/physical-plan/src/sort_pushdown.rs | 120 ++
datafusion/physical-plan/src/sorts/cursor.rs | 4 +-
datafusion/physical-plan/src/sorts/merge.rs | 4 +-
.../physical-plan/src/sorts/multi_level_merge.rs | 8 +-
datafusion/physical-plan/src/sorts/partial_sort.rs | 159 +-
datafusion/physical-plan/src/sorts/sort.rs | 98 +-
.../src/sorts/sort_preserving_merge.rs | 174 +-
datafusion/physical-plan/src/sorts/stream.rs | 6 +-
.../physical-plan/src/sorts/streaming_merge.rs | 6 +-
.../src/spill/in_progress_spill_file.rs | 2 +-
datafusion/physical-plan/src/spill/mod.rs | 214 +-
.../physical-plan/src/spill/spill_manager.rs | 6 +-
datafusion/physical-plan/src/spill/spill_pool.rs | 67 +-
datafusion/physical-plan/src/stream.rs | 4 +-
datafusion/physical-plan/src/streaming.rs | 8 +-
datafusion/physical-plan/src/test.rs | 13 +-
datafusion/physical-plan/src/test/exec.rs | 12 +-
datafusion/physical-plan/src/topk/mod.rs | 18 +-
datafusion/physical-plan/src/tree_node.rs | 4 +-
datafusion/physical-plan/src/union.rs | 62 +-
datafusion/physical-plan/src/unnest.rs | 64 +-
.../src/windows/bounded_window_agg_exec.rs | 144 +-
datafusion/physical-plan/src/windows/mod.rs | 58 +-
.../physical-plan/src/windows/window_agg_exec.rs | 4 +-
datafusion/physical-plan/src/work_table.rs | 7 +-
.../proto-common/proto/datafusion_common.proto | 6 +
datafusion/proto-common/src/common.rs | 2 +-
datafusion/proto-common/src/from_proto/mod.rs | 20 +-
datafusion/proto-common/src/generated/mod.rs | 1 +
datafusion/proto-common/src/generated/pbjson.rs | 76 +
datafusion/proto-common/src/generated/prost.rs | 11 +
datafusion/proto-common/src/lib.rs | 3 +-
datafusion/proto-common/src/to_proto/mod.rs | 14 +-
datafusion/proto/Cargo.toml | 1 +
datafusion/proto/proto/datafusion.proto | 19 +
datafusion/proto/src/bytes/mod.rs | 8 +-
datafusion/proto/src/bytes/registry.rs | 29 +-
datafusion/proto/src/common.rs | 4 +-
datafusion/proto/src/generated/datafusion.rs | 1 +
.../proto/src/generated/datafusion_proto_common.rs | 11 +
datafusion/proto/src/generated/mod.rs | 2 +
datafusion/proto/src/generated/pbjson.rs | 376 ++++
datafusion/proto/src/generated/prost.rs | 29 +-
datafusion/proto/src/lib.rs | 3 +-
datafusion/proto/src/logical_plan/file_formats.rs | 23 +-
datafusion/proto/src/logical_plan/from_proto.rs | 19 +-
datafusion/proto/src/logical_plan/mod.rs | 83 +-
datafusion/proto/src/logical_plan/to_proto.rs | 45 +-
datafusion/proto/src/physical_plan/from_proto.rs | 45 +-
datafusion/proto/src/physical_plan/mod.rs | 137 +-
datafusion/proto/src/physical_plan/to_proto.rs | 63 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 42 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 210 +-
datafusion/proto/tests/cases/serialize.rs | 6 +-
datafusion/pruning/src/file_pruner.rs | 131 +-
datafusion/pruning/src/lib.rs | 7 +-
datafusion/pruning/src/pruning_predicate.rs | 373 +++-
datafusion/session/src/lib.rs | 3 +-
datafusion/session/src/session.rs | 4 +-
datafusion/spark/Cargo.toml | 2 +
datafusion/spark/benches/char.rs | 2 +-
datafusion/spark/src/function/aggregate/avg.rs | 6 +-
datafusion/spark/src/function/aggregate/mod.rs | 12 +-
datafusion/spark/src/function/aggregate/try_sum.rs | 660 ++++++
datafusion/spark/src/function/array/shuffle.rs | 66 +-
datafusion/spark/src/function/array/spark_array.rs | 135 +-
.../spark/src/function/bitmap/bitmap_count.rs | 73 +-
datafusion/spark/src/function/bitwise/bit_count.rs | 60 +-
datafusion/spark/src/function/bitwise/bit_get.rs | 62 +-
datafusion/spark/src/function/bitwise/bit_shift.rs | 78 +-
.../spark/src/function/bitwise/bitwise_not.rs | 130 +-
datafusion/spark/src/function/conditional/if.rs | 6 +-
datafusion/spark/src/function/datetime/date_add.rs | 92 +-
datafusion/spark/src/function/datetime/date_sub.rs | 84 +-
datafusion/spark/src/function/datetime/last_day.rs | 82 +-
.../src/function/datetime/make_dt_interval.rs | 175 +-
.../spark/src/function/datetime/make_interval.rs | 89 +-
datafusion/spark/src/function/datetime/next_day.rs | 89 +-
datafusion/spark/src/function/error_utils.rs | 6 +-
datafusion/spark/src/function/hash/crc32.rs | 4 +-
datafusion/spark/src/function/hash/sha1.rs | 63 +-
datafusion/spark/src/function/hash/sha2.rs | 35 +-
.../spark/src/function/map/map_from_arrays.rs | 88 +-
.../spark/src/function/map/map_from_entries.rs | 103 +-
datafusion/spark/src/function/map/utils.rs | 11 +-
datafusion/spark/src/function/math/abs.rs | 2 +-
datafusion/spark/src/function/math/factorial.rs | 4 +-
datafusion/spark/src/function/math/hex.rs | 99 +-
datafusion/spark/src/function/math/mod.rs | 6 +-
datafusion/spark/src/function/math/modulus.rs | 12 +-
datafusion/spark/src/function/math/rint.rs | 5 +-
datafusion/spark/src/function/math/width_bucket.rs | 149 +-
datafusion/spark/src/function/string/char.rs | 63 +-
datafusion/spark/src/function/string/concat.rs | 77 +-
datafusion/spark/src/function/string/elt.rs | 67 +-
.../spark/src/function/string/format_string.rs | 24 +-
datafusion/spark/src/function/string/ilike.rs | 90 +-
datafusion/spark/src/function/string/length.rs | 54 +-
datafusion/spark/src/function/string/like.rs | 2 +-
datafusion/spark/src/function/string/luhn_check.rs | 2 +-
datafusion/spark/src/function/url/mod.rs | 29 +-
datafusion/spark/src/function/url/parse_url.rs | 11 +-
datafusion/spark/src/function/url/try_parse_url.rs | 2 +-
.../url/{try_parse_url.rs => try_url_decode.rs} | 64 +-
datafusion/spark/src/function/url/url_decode.rs | 261 +++
datafusion/spark/src/function/url/url_encode.rs | 131 ++
datafusion/spark/src/lib.rs | 7 +-
datafusion/sql/Cargo.toml | 2 +-
datafusion/sql/examples/sql.rs | 6 +-
datafusion/sql/src/cte.rs | 15 +-
datafusion/sql/src/expr/binary_op.rs | 2 +-
datafusion/sql/src/expr/function.rs | 78 +-
datafusion/sql/src/expr/identifier.rs | 86 +-
datafusion/sql/src/expr/mod.rs | 26 +-
datafusion/sql/src/expr/order_by.rs | 2 +-
datafusion/sql/src/expr/subquery.rs | 18 +-
datafusion/sql/src/expr/substring.rs | 10 +-
datafusion/sql/src/expr/unary_op.rs | 11 +-
datafusion/sql/src/expr/value.rs | 35 +-
datafusion/sql/src/lib.rs | 3 +-
datafusion/sql/src/parser.rs | 65 +-
datafusion/sql/src/planner.rs | 22 +-
datafusion/sql/src/query.rs | 33 +-
datafusion/sql/src/relation/join.rs | 2 +-
datafusion/sql/src/relation/mod.rs | 123 +-
datafusion/sql/src/select.rs | 206 +-
datafusion/sql/src/set_expr.rs | 2 +-
datafusion/sql/src/statement.rs | 100 +-
datafusion/sql/src/unparser/ast.rs | 20 +-
datafusion/sql/src/unparser/dialect.rs | 4 +-
datafusion/sql/src/unparser/expr.rs | 53 +-
datafusion/sql/src/unparser/extension_unparser.rs | 4 +-
datafusion/sql/src/unparser/plan.rs | 160 +-
datafusion/sql/src/unparser/rewrite.rs | 13 +-
datafusion/sql/src/unparser/utils.rs | 48 +-
datafusion/sql/src/utils.rs | 67 +-
datafusion/sql/tests/cases/collection.rs | 10 +-
datafusion/sql/tests/cases/diagnostic.rs | 3 +-
datafusion/sql/tests/cases/params.rs | 87 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 204 +-
datafusion/sql/tests/common/mod.rs | 2 +-
datafusion/sql/tests/sql_integration.rs | 2280 ++++++++++----------
datafusion/sqllogictest/Cargo.toml | 2 +-
datafusion/sqllogictest/bin/postgres_container.rs | 10 +-
datafusion/sqllogictest/bin/sqllogictests.rs | 41 +-
datafusion/sqllogictest/src/engines/conversion.rs | 5 +-
.../src/engines/datafusion_engine/runner.rs | 2 +-
.../runner.rs | 3 +-
datafusion/sqllogictest/src/engines/mod.rs | 4 +-
.../src/engines/postgres_engine/mod.rs | 4 +-
datafusion/sqllogictest/src/filters.rs | 19 +-
datafusion/sqllogictest/src/lib.rs | 4 +-
datafusion/sqllogictest/src/test_context.rs | 62 +-
datafusion/sqllogictest/src/util.rs | 2 +-
.../test_files/agg_func_substitute.slt | 27 +-
datafusion/sqllogictest/test_files/aggregate.slt | 230 +-
.../test_files/aggregate_repartition.slt | 16 +-
.../sqllogictest/test_files/aggregates_topk.slt | 49 +-
datafusion/sqllogictest/test_files/array.slt | 54 +-
.../sqllogictest/test_files/arrow_typeof.slt | 10 +-
datafusion/sqllogictest/test_files/async_udf.slt | 15 +-
datafusion/sqllogictest/test_files/case.slt | 134 ++
datafusion/sqllogictest/test_files/cast.slt | 36 +
datafusion/sqllogictest/test_files/coalesce.slt | 6 +-
.../sqllogictest/test_files/count_star_rule.slt | 14 +-
.../test_files/create_external_table.slt | 2 +-
datafusion/sqllogictest/test_files/cte.slt | 92 +-
datafusion/sqllogictest/test_files/dates.slt | 102 +
datafusion/sqllogictest/test_files/decimal.slt | 34 +-
datafusion/sqllogictest/test_files/distinct_on.slt | 9 +-
.../test_files/dynamic_filter_pushdown_config.slt | 471 +++-
datafusion/sqllogictest/test_files/encoding.slt | 13 +
datafusion/sqllogictest/test_files/errors.slt | 5 +
datafusion/sqllogictest/test_files/explain.slt | 30 +-
.../sqllogictest/test_files/explain_tree.slt | 360 ++--
datafusion/sqllogictest/test_files/expr.slt | 7 +-
datafusion/sqllogictest/test_files/group_by.slt | 218 +-
datafusion/sqllogictest/test_files/grouping.slt | 12 +
.../sqllogictest/test_files/information_schema.slt | 18 +-
datafusion/sqllogictest/test_files/insert.slt | 15 +-
.../sqllogictest/test_files/insert_to_external.slt | 10 +-
datafusion/sqllogictest/test_files/join.slt.part | 85 +-
.../test_files/join_disable_repartition_joins.slt | 20 +-
.../test_files/join_is_not_distinct_from.slt | 93 +-
datafusion/sqllogictest/test_files/joins.slt | 772 ++++---
datafusion/sqllogictest/test_files/limit.slt | 16 +-
.../test_files/listing_table_statistics.slt | 2 +-
datafusion/sqllogictest/test_files/map.slt | 24 +-
datafusion/sqllogictest/test_files/math.slt | 101 +-
datafusion/sqllogictest/test_files/metadata.slt | 49 +
.../test_files/monotonic_projection_test.slt | 8 +-
datafusion/sqllogictest/test_files/order.slt | 57 +-
.../test_files/parquet_sorted_statistics.slt | 3 +-
.../sqllogictest/test_files/parquet_statistics.slt | 12 +-
datafusion/sqllogictest/test_files/predicates.slt | 43 +-
.../test_files/preserve_file_partitioning.slt | 596 +++++
.../sqllogictest/test_files/push_down_filter.slt | 85 +-
datafusion/sqllogictest/test_files/qualify.slt | 25 +-
datafusion/sqllogictest/test_files/repartition.slt | 14 +-
datafusion/sqllogictest/test_files/scalar.slt | 14 +-
datafusion/sqllogictest/test_files/select.slt | 17 +-
.../sqllogictest/test_files/set_variable.slt | 14 +
.../sqllogictest/test_files/slt_features.slt | 2 +-
.../test_files/spark/aggregate/try_sum.slt | 140 ++
.../test_files/spark/bitwise/bit_count.slt | 2 +-
.../test_files/spark/datetime/make_dt_interval.slt | 3 +-
.../sqllogictest/test_files/spark/math/hex.slt | 15 +
.../url/try_url_decode.slt} | 65 +-
.../test_files/spark/url/url_decode.slt | 60 +-
.../test_files/spark/url/url_encode.slt | 24 +-
.../test_files/string/string_literal.slt | 39 +
.../sqllogictest/test_files/string/string_view.slt | 2 +-
datafusion/sqllogictest/test_files/struct.slt | 38 +-
datafusion/sqllogictest/test_files/subquery.slt | 99 +-
.../sqllogictest/test_files/subquery_sort.slt | 9 +-
datafusion/sqllogictest/test_files/timestamps.slt | 447 +++-
datafusion/sqllogictest/test_files/topk.slt | 5 +-
.../sqllogictest/test_files/tpch/plans/q1.slt.part | 11 +-
.../test_files/tpch/plans/q10.slt.part | 44 +-
.../test_files/tpch/plans/q11.slt.part | 69 +-
.../test_files/tpch/plans/q12.slt.part | 20 +-
.../test_files/tpch/plans/q13.slt.part | 24 +-
.../test_files/tpch/plans/q14.slt.part | 15 +-
.../test_files/tpch/plans/q15.slt.part | 43 +-
.../test_files/tpch/plans/q16.slt.part | 44 +-
.../test_files/tpch/plans/q17.slt.part | 31 +-
.../test_files/tpch/plans/q18.slt.part | 43 +-
.../test_files/tpch/plans/q19.slt.part | 19 +-
.../sqllogictest/test_files/tpch/plans/q2.slt.part | 113 +-
.../test_files/tpch/plans/q20.slt.part | 62 +-
.../test_files/tpch/plans/q21.slt.part | 69 +-
.../test_files/tpch/plans/q22.slt.part | 44 +-
.../sqllogictest/test_files/tpch/plans/q3.slt.part | 37 +-
.../sqllogictest/test_files/tpch/plans/q4.slt.part | 22 +-
.../sqllogictest/test_files/tpch/plans/q5.slt.part | 68 +-
.../sqllogictest/test_files/tpch/plans/q7.slt.part | 74 +-
.../sqllogictest/test_files/tpch/plans/q8.slt.part | 96 +-
.../sqllogictest/test_files/tpch/plans/q9.slt.part | 68 +-
datafusion/sqllogictest/test_files/union.slt | 147 +-
datafusion/sqllogictest/test_files/unnest.slt | 32 +-
datafusion/sqllogictest/test_files/window.slt | 240 +--
.../sqllogictest/test_files/window_limits.slt | 10 +-
datafusion/substrait/Cargo.toml | 4 +-
datafusion/substrait/src/extensions.rs | 6 +-
datafusion/substrait/src/lib.rs | 3 +-
.../consumer/expr/aggregate_function.rs | 6 +-
.../src/logical_plan/consumer/expr/cast.rs | 4 +-
.../logical_plan/consumer/expr/field_reference.rs | 4 +-
.../consumer/expr/function_arguments.rs | 4 +-
.../src/logical_plan/consumer/expr/literal.rs | 48 +-
.../src/logical_plan/consumer/expr/mod.rs | 26 +-
.../logical_plan/consumer/expr/scalar_function.rs | 133 +-
.../logical_plan/consumer/expr/singular_or_list.rs | 4 +-
.../src/logical_plan/consumer/expr/subquery.rs | 8 +-
.../logical_plan/consumer/expr/window_function.rs | 10 +-
.../substrait/src/logical_plan/consumer/plan.rs | 67 +-
.../src/logical_plan/consumer/rel/aggregate_rel.rs | 8 +-
.../src/logical_plan/consumer/rel/exchange_rel.rs | 4 +-
.../src/logical_plan/consumer/rel/fetch_rel.rs | 6 +-
.../src/logical_plan/consumer/rel/join_rel.rs | 6 +-
.../substrait/src/logical_plan/consumer/rel/mod.rs | 6 +-
.../src/logical_plan/consumer/rel/project_rel.rs | 4 +-
.../src/logical_plan/consumer/rel/read_rel.rs | 124 +-
.../src/logical_plan/consumer/rel/set_rel.rs | 5 +-
.../src/logical_plan/consumer/rel/sort_rel.rs | 2 +-
.../logical_plan/consumer/substrait_consumer.rs | 12 +-
.../substrait/src/logical_plan/consumer/types.rs | 40 +-
.../substrait/src/logical_plan/consumer/utils.rs | 37 +-
.../producer/expr/aggregate_function.rs | 2 +-
.../src/logical_plan/producer/expr/cast.rs | 6 +-
.../logical_plan/producer/expr/field_reference.rs | 6 +-
.../src/logical_plan/producer/expr/if_then.rs | 2 +-
.../src/logical_plan/producer/expr/literal.rs | 8 +-
.../src/logical_plan/producer/expr/mod.rs | 8 +-
.../logical_plan/producer/expr/scalar_function.rs | 14 +-
.../logical_plan/producer/expr/singular_or_list.rs | 2 +-
.../src/logical_plan/producer/expr/subquery.rs | 2 +-
.../logical_plan/producer/expr/window_function.rs | 10 +-
.../substrait/src/logical_plan/producer/plan.rs | 6 +-
.../src/logical_plan/producer/rel/aggregate_rel.rs | 40 +-
.../src/logical_plan/producer/rel/exchange_rel.rs | 6 +-
.../src/logical_plan/producer/rel/fetch_rel.rs | 2 +-
.../src/logical_plan/producer/rel/join.rs | 6 +-
.../src/logical_plan/producer/rel/project_rel.rs | 4 +-
.../src/logical_plan/producer/rel/read_rel.rs | 169 +-
.../src/logical_plan/producer/rel/set_rel.rs | 2 +-
.../src/logical_plan/producer/rel/sort_rel.rs | 4 +-
.../logical_plan/producer/substrait_producer.rs | 14 +-
.../substrait/src/logical_plan/producer/types.rs | 10 +-
.../substrait/src/logical_plan/producer/utils.rs | 4 +-
datafusion/substrait/src/physical_plan/consumer.rs | 26 +-
datafusion/substrait/src/physical_plan/producer.rs | 157 +-
.../substrait/tests/cases/aggregation_tests.rs | 16 +-
.../substrait/tests/cases/consumer_integration.rs | 420 ++--
.../substrait/tests/cases/emit_kind_tests.rs | 44 +-
datafusion/substrait/tests/cases/function_test.rs | 8 +-
datafusion/substrait/tests/cases/logical_plans.rs | 65 +-
.../tests/cases/roundtrip_logical_plan.rs | 316 ++-
.../tests/cases/roundtrip_physical_plan.rs | 2 +-
datafusion/substrait/tests/cases/serialize.rs | 20 +-
.../substrait/tests/cases/substrait_validations.rs | 24 +-
datafusion/substrait/tests/utils.rs | 22 +-
datafusion/wasmtest/Cargo.toml | 2 +-
.../wasmtest/datafusion-wasm-app/package-lock.json | 102 +-
datafusion/wasmtest/src/lib.rs | 4 +-
dev/depcheck/Cargo.toml | 3 +-
.../depcheck/rust-toolchain.toml | 2 +-
dev/depcheck/src/main.rs | 2 +-
dev/rust_lint.sh | 14 +-
docs/requirements.txt | 2 +-
docs/source/contributor-guide/governance.md | 5 +-
docs/source/contributor-guide/howtos.md | 8 +-
docs/source/contributor-guide/inviting.md | 2 +-
docs/source/index.rst | 1 +
.../library-user-guide/custom-table-providers.md | 2 +-
.../library-user-guide/extending-operators.md | 7 +-
docs/source/library-user-guide/extending-sql.md | 339 +++
.../library-user-guide/functions/adding-udfs.md | 28 +-
docs/source/library-user-guide/query-optimizer.md | 20 +-
docs/source/library-user-guide/upgrading.md | 369 +++-
.../library-user-guide/using-the-dataframe-api.md | 2 +-
.../library-user-guide/working-with-exprs.md | 8 +-
docs/source/user-guide/cli/functions.md | 32 +
docs/source/user-guide/configs.md | 10 +-
docs/source/user-guide/crate-configuration.md | 30 +
docs/source/user-guide/expressions.md | 2 +-
docs/source/user-guide/introduction.md | 3 +
docs/source/user-guide/metrics.md | 10 +-
docs/source/user-guide/sql/ddl.md | 2 +-
docs/source/user-guide/sql/scalar_functions.md | 120 +-
rust-toolchain.toml | 2 +-
rustfmt.toml | 2 +-
test-utils/src/array_gen/binary.rs | 2 +-
test-utils/src/array_gen/boolean.rs | 2 +-
test-utils/src/array_gen/decimal.rs | 2 +-
test-utils/src/array_gen/primitive.rs | 4 +-
test-utils/src/array_gen/random_data.rs | 16 +-
test-utils/src/array_gen/string.rs | 2 +-
test-utils/src/string_gen.rs | 2 +-
test-utils/src/tpcds.rs | 2 +-
typos.toml | 6 +-
1260 files changed, 55414 insertions(+), 24755 deletions(-)
copy datafusion/proto-common/regen.sh => benchmarks/compare_tpcds.sh (52%)
copy datafusion/proto-common/regen.sh => benchmarks/compare_tpch.sh (55%)
copy benchmarks/queries/clickbench/queries/{ => sorted_data}/q0.sql (75%)
delete mode 100644 benchmarks/src/bin/tpch.rs
create mode 100644 benchmarks/src/smj.rs
copy {datafusion/core/tests/catalog => benchmarks/src/tpcds}/mod.rs (96%)
create mode 100644 benchmarks/src/tpcds/run.rs
delete mode 100644 benchmarks/src/tpch/convert.rs
create mode 100755 ci/scripts/doc_prettier_check.sh
copy ci/scripts/{rust_docs.sh => typos_check.sh} (84%)
rename datafusion-examples/examples/external_dependency/{dataframe-to-s3.rs =>
dataframe_to_s3.rs} (87%)
create mode 100644 datafusion-examples/examples/external_dependency/main.rs
rename datafusion-examples/examples/external_dependency/{query-aws-s3.rs =>
query_aws_s3.rs} (90%)
create mode 100644 datafusion-examples/examples/relation_planner/main.rs
create mode 100644
datafusion-examples/examples/relation_planner/match_recognize.rs
create mode 100644
datafusion-examples/examples/relation_planner/pivot_unpivot.rs
create mode 100644
datafusion-examples/examples/relation_planner/table_sample.rs
create mode 100644 datafusion/common/benches/with_hashes.rs
create mode 100644 datafusion/common/src/display/human_readable.rs
create mode 100644 datafusion/core/benches/preserve_file_partitioning.rs
create mode 100644 datafusion/core/tests/data/recursive_cte/closure.csv
create mode 100644 datafusion/core/tests/physical_optimizer/pushdown_sort.rs
create mode 100644 datafusion/core/tests/user_defined/relation_planner.rs
create mode 100644 datafusion/datasource-parquet/src/sort.rs
create mode 100644 datafusion/datasource/src/projection.rs
copy datafusion/execution/src/cache/{cache_unit.rs => file_metadata_cache.rs}
(78%)
create mode 100644 datafusion/execution/src/cache/list_files_cache.rs
rename datafusion/{physical-plan => execution}/src/metrics/baseline.rs (97%)
rename datafusion/{physical-plan => execution}/src/metrics/builder.rs (99%)
rename datafusion/{physical-plan => execution}/src/metrics/custom.rs (98%)
rename datafusion/{physical-plan => execution}/src/metrics/mod.rs (96%)
rename datafusion/{physical-plan => execution}/src/metrics/value.rs (99%)
copy datafusion/{core/tests/fuzz_cases/equivalence =>
ffi/src/execution}/mod.rs (86%)
create mode 100644 datafusion/ffi/src/execution/task_ctx.rs
create mode 100644 datafusion/ffi/src/execution/task_ctx_provider.rs
create mode 100644 datafusion/ffi/src/expr/columnar_value.rs
create mode 100644 datafusion/ffi/src/expr/distribution.rs
create mode 100644 datafusion/ffi/src/expr/expr_properties.rs
create mode 100644 datafusion/ffi/src/expr/interval.rs
copy datafusion/{core/tests/catalog => ffi/src/expr}/mod.rs (90%)
create mode 100644 datafusion/ffi/src/physical_expr/mod.rs
create mode 100644 datafusion/ffi/src/physical_expr/partitioning.rs
create mode 100644 datafusion/ffi/src/physical_expr/sort.rs
create mode 100644 datafusion/ffi/src/proto/logical_extension_codec.rs
copy datafusion/{core/tests/catalog => ffi/src/proto}/mod.rs (92%)
create mode 100644 datafusion/ffi/src/proto/physical_extension_codec.rs
rename datafusion/ffi/src/{session_config.rs => session/config.rs} (63%)
create mode 100644 datafusion/ffi/src/session/mod.rs
create mode 100644 datafusion/functions-nested/benches/array_has.rs
create mode 100644 datafusion/functions-nested/benches/array_slice.rs
create mode 100644 datafusion/functions/src/crypto/sha.rs
delete mode 100644 datafusion/functions/src/crypto/sha224.rs
delete mode 100644 datafusion/functions/src/crypto/sha256.rs
delete mode 100644 datafusion/functions/src/crypto/sha384.rs
delete mode 100644 datafusion/functions/src/crypto/sha512.rs
create mode 100644 datafusion/functions/src/datetime/make_time.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/boolean_lookup_table.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/bytes_like_lookup_table.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/mod.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/primitive_lookup_table.rs
create mode 100644 datafusion/physical-expr/src/simplifier/const_evaluator.rs
create mode 100644 datafusion/physical-expr/src/simplifier/not.rs
create mode 100644 datafusion/physical-optimizer/src/pushdown_sort.rs
create mode 100644
datafusion/physical-plan/src/joins/hash_join/inlist_builder.rs
copy datafusion/{core/src/bin/print_config_docs.rs =>
physical-plan/src/metrics.rs} (84%)
create mode 100644 datafusion/physical-plan/src/sort_pushdown.rs
create mode 100644 datafusion/proto/src/generated/datafusion.rs
create mode 100644 datafusion/spark/src/function/aggregate/try_sum.rs
copy datafusion/spark/src/function/url/{try_parse_url.rs => try_url_decode.rs}
(55%)
create mode 100644 datafusion/spark/src/function/url/url_decode.rs
create mode 100644 datafusion/spark/src/function/url/url_encode.rs
create mode 100644
datafusion/sqllogictest/test_files/preserve_file_partitioning.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/aggregate/try_sum.slt
copy datafusion/sqllogictest/test_files/{slt_features.slt =>
spark/url/try_url_decode.slt} (52%)
copy rust-toolchain.toml => dev/depcheck/rust-toolchain.toml (98%)
create mode 100644 docs/source/library-user-guide/extending-sql.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
