This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/pip/docs/sphinx-9.0.4
in repository https://gitbox.apache.org/repos/asf/datafusion.git
discard cc3630ec73 Merge branch 'main' into dependabot/pip/docs/sphinx-9.0.4
discard ba65ca1084 chore(deps): bump sphinx from 8.2.3 to 9.0.4 in /docs
add adaed426a8 Arc partition values in TableSchema (#19137)
add 0812d2731a feat: Move DefaultMetadataCache into its own module (#19125)
add 662a3bad64 fix: inverted null_percent logic in in_list benchmark
(#19204)
add cde6dfab1c Add sorted data benchmark. (#19042)
add f2b476f390 Refactor PhysicalExprSimplfier to &self instead of &mut
self (#19212)
add 3fabee7705 Revert adding PhysicalOptimizerRule::optimize_plan (#19186)
add 4504d941b6 chore(deps): bump uuid from 1.18.1 to 1.19.0 (#19199)
add af3ab52a75 chore(deps): bump async-compression from 0.4.34 to 0.4.35
(#19201)
add dcc09c3450 chore(deps): bump taiki-e/install-action from 2.62.62 to
2.62.63 (#19198)
add 7d958b0408 chore(deps): bump tracing-subscriber from 0.3.20 to 0.3.22
(#19200)
add 71a6982859 chore(deps): bump wasm-bindgen-test from 0.3.55 to 0.3.56
(#19202)
add 55a38d4a6d feat: Add `remove_optimizer_rule` to `SessionContext`
(#19209)
add ab7fe0eb51 bench: add dedicated Utf8View benchmarks for InList (#19211)
add 83736efc4a Fix PruningPredicate interaction with
DynamicFilterPhysicalExpr that references partition columns (#19129)
add ad9b779b6d Implement physical and logical codecs in FFI (#19079)
add 4ddee14ee8 refactor: Refactor spark width bucket signature away from
user defined (#19065)
add 2626fc2024 fix: Ensure column names do not change with
`expand_views_at_output` (#19019)
add 4fb36b23d9 fix: bitmap_count should report nullability correctly
(#19195)
add 1a6df66e21 fix: bit_count function to report nullability correctly
(#19197)
add ea9f354123 fix: derive custom nullability for spark `bit_shift`
(#19222)
add 7ea5066fe4 Sort Merge Join: Reduce batch concatenation, use
`BatchCoalescer`, new benchmarks (TPC-H Q21 SMJ up to ~4000x faster) (#18875)
add a30cf37099 Add relation planner extension support to customize SQL
planning (#17843)
add 20870c18a4 feat: integrate batch coalescer with repartition exec
(#19002)
add c0e8bb501a Push down InList or hash table references from HashJoinExec
depending on the size of the build side (#18393)
add 21a16e44bc Add additional tests for InListExpr (#19050)
add 7ac40d796a chore(deps): bump taiki-e/install-action from 2.62.63 to
2.62.64 (#19226)
add dc786138cf Move partition handling out of PhysicalExprAdapter (#19128)
add cd160de0bf fix: spark elt custom nullability (#19207)
add dc6a71235b Use strum in the examples (#19126) (#19205)
add d85595e032 perf: optimize CASE WHEN lookup table (2.5-22.5 times
faster) (#18183)
add e8a0829b7a Allow Logical expression ScalarVariable to represent an
extension type or metadata (#18243)
add dc4e3ab473 feat: Implement the `statistics_cache` function (#19054)
add ca67edcf99 [Proto]: Serialization support for `AsyncFuncExec` (#19118)
add 26af02a48b chore: add test case for decimal overflow (#19255)
add 79a2f5e110 chore(deps): bump taiki-e/install-action from 2.62.64 to
2.62.65 (#19251)
add 2a08013af3 chore: update 6 crates to rust edition 2024 (#19196)
add 021188e719 Push down projection expressions into ParquetOpener (#19111)
add 321401ce62 fix: `array_remove`/`array_remove_n`/`array_remove_all` not
using the same nullability as the input (#19259)
add c1aa1b530a Track column sizes in Statistics; propagate through
projections (#19113)
add 39a1f75751 Implement FFI_Session (#19223)
add 89a9c9d06d Feat: Add an option for fast tests by gating slow tests to
extended_tests feature (#19237)
add 5496c30431 fix: typo in sql/ddl (#19276)
add e914935660 chore: enforce clippy::allow_attributes for 7 crates
(#19133)
add bde16083ad feat: Preserve File Partitioning From File Scans (#19124)
add 58345f94af Improve ProjectionExpr documentation and comments (#19263)
add 85d8a88c10 add specialized InList implementations for common scalar
types (#18832)
add c8add5c1d9 Update README occording to the new examples (#18529)
(#19257)
add 044a4a7369 Add make_time function (#19183)
add 33b3f02164 dev: Add CI doc prettier check to local `rust_lint.sh`
(#19254)
add a3b3eb5582 bug: Eliminate dead round-robin insertion in enforce
distribution (#19132)
add f9c030a5f5 Automatically download tpcds benchmark data to the right
place (#19244)
add 86011519b5 fix: flaky cache test (#19140)
add 10c933cc76 [datafusion-spark]: Refactor hex's signature away from
user_defined (#19235)
add 06f27e94d9 fix : correct nullability propagation for spark.bitwise_not
(#19224)
add aebae990dc fix: Add custom nullability for Spark ILIKE function
(#19206)
add 96ddd558f8 fix: derive custom nullability for spark `map_from_arrays`
(#19275)
add 18e4a0caf7 fix: derive custom nullability for spark map_from_entries
(#19274)
add 2c6f6d3696 fix: derive custom nullable for spark `make_dt_interval`
(#19236)
add c4ca946caf fix: derive custome nullable for the spark last_day (#19232)
add 49cfee05c2 fix: derive custom nullable for spark `date_sub` (#19225)
add 8458946419 added custom nullability for char (#19268)
add 5a01e68643 Update to_date udf function to support a consistent set of
argument types (#19134)
add d61f1a7150 replace HashTableLookupExpr with lit(true) in proto
serialization (#19300)
add fedddbcae4 chore: fix return_field_from_args doc (#19307)
add 4088192fc8 chore: enforce clippy::allow_attributes for
spark,sql,sustrait (#19309)
add 94d9c8e68c Simplify make_date & fix null handling (#19296)
add 033116e8a2 fix: Fix a few minor issues with join metrics (#19283)
add 9163feed6b fix: derive nullability for spark `bit_get` (#19220)
add 16fa35fa25 Allow base64 encoding of fixedsizebinary arrays (#18950)
add befaf93a15 chore: update 11 crates to Rust 2024 edition (#19258)
add ec11f42508 feat: Add constant column extraction and rewriting for
projections in ParquetOpener (#19136)
add dbf9265e5c fix: pow() with integer base and negative float exponent
returns error (#19303)
add 2bea796897 Minor: remove unnecessary unit tests for fixed size binary
(#19318)
add 899a762230 Populate partition column statistics for PartitionedFile
(#19284)
add 7c05b208e1 refactor: move metrics module to `datafusion-common` crate
(#19247)
add 9d4fe15895 Add library user guide for extending SQL syntax (#19265)
add b828fbfa54 chore(deps): bump taiki-e/install-action from 2.62.65 to
2.62.67 (#19295)
add 58377bf2e3 chore(deps): bump ctor from 0.6.1 to 0.6.3 (#19328)
add fc8824011b Refactor `power()` signature away from user defined (#18968)
add 933657e2c8 feat: Support sliding window queries for MedianAccumulator
by implementing `retract_batch` (#19278)
add 02c647ae1e chore: enforce `clippy::allow_attributes` for optimizer and
macros (#19310)
add 3a67d86c94 fix(concat): correct nullability inference (nullable only
if all arguments nullable) (#19189)
add 266a7d060e chore(deps): bump taiki-e/install-action from 2.62.67 to
2.63.3 (#19349)
add ea8087d77e chore(deps): bump clap from 4.5.50 to 4.5.53 (#19326)
add 199cf06d20 chore(deps): bump insta from 1.43.2 to 1.44.3 (#19327)
add 79cfe8e921 Add runtime config options for `list_files_cache_limit` and
`list_files_cache_ttl` (#19108)
add 89af694117 remove repartition exec from coalesce batches optimizer
(#19239)
add be0cf05f93 minor: cleanup unnecessary config in `decimal.slt` (#19352)
add c53a44891b Fix panic for `GROUPING SETS(())` and handle empty-grouping
aggregates (#19252)
add 775277ae3c Update datafusion-core crate to Rust 2024 edition (#19332)
add efd793ba3e Minor: clean up titles and links n extending operators and
optimizer pages (#19317)
add 1f26716280 Update 4 crates to rust 2024 edition (#19357)
add 50d20dd553 preserve Field metadata in first_value/last_value (#19335)
add 3a41cc6078 Establish the high level API for sort pushdown and the
optimizer rule and support reverse files and row groups (#19064)
add b806117bfa Fix flaky SpillPool channel test by synchronizing reader
and writer tasks (#19110)
add 2b67ba81c6 [minor] Upgrade rust version (#19363)
add 401345b6fd Minor: fix cargo fmt (#19368)
add 7900cd6e98 chore: enforce clippy::allow_attributes for proto, pruning,
session (#19350)
add 59dcc362ca Update remaining crates to rust 2024 edition (#19361)
add 2e95f2673f Minor: Make `ProjectionExpr::new` easier to use with
constants (#19343)
add fe11ad6b46 fix: Added nullable return from date_add(#19151) (#19229)
add 3879b24503 fix: spark sha1 nullability reporting (#19242)
add 1e4bd7565b Feat: DefaultListFilesCache prefix-aware for partition
pruning optimization (#19298)
add 14cd71eaba feat: add compression level configuration for JSON/CSV
writers (#18954)
add 4e7bba4909 Extend in_list benchmark coverage (#19376)
add d68b629dc6 Add hashing microbenchmark `with_hashes` (#19373)
add cb3fa1c2c7 fix: derive custom nullability for the spark `next_day`
(#19253)
add 0a0416dbfe Move `newlines_in_values` from `FileScanConfig` to
`CsvSource` (#19313)
add 2c9077a8bf [datafusion-cli] Implement average LIST duration for object
store profiling (#19127)
add 8120da40a9 chore(deps): bump taiki-e/install-action from 2.63.3 to
2.64.0 (#19382)
add ead8209803 update insta snapshots (#19381)
add d59ebac9f3 feat(spark): implement Spark `try_sum` function (#18569)
add c2747eb109 feat: Support log for Decimal32 and Decimal64 (#18999)
add cbf33d1ee9 Fix regression for negative-scale decimal128 in log (#19315)
add d493f3d441 Add Decimal support to Ceil and Floor (#18979)
add 8550010bd5 Fix input handling for encoding functions & various
refactors (#18754)
add 75d2473ba7 Remove SchemaAdapter (#19345)
add 887aa9f8c6 fix: preserve ListFilesCache TTL when not set in config
(#19401)
add 91cfb69903 feat(proto): Add protobuf serialization for HashExpr
(#19379)
add 2e3707e380 fix: projection for `CooperativeExec` and
`CoalesceBatchesExec` (#19400)
add b3d2cb622c Fix ORDER BY positional reference regression with aliased
aggregates (#19412)
add 5419ff5902 feat: hash partitioning satisfies subset (#19304)
add 8cc8c11de0 Optimize muti-column grouping with StringView/ByteView
(option 2) - 25% faster (#19413)
add 6fa9c1ad11 Optimize hashing for StringView and ByteView (15-70%
faster) (#19374)
add eb30c19b30 Implement disk spilling for all grouping ordering modes in
GroupedHashAggregateStream (#19287)
add 32e6fe887a feat: update FFI TableProvider and ExecutionPlan to use FFI
Session and TaskContext (#19281)
add d8e68a404a refactor: add ParquetOpenerBuilder to reduce test code
duplication (#19405)
add 4249e4ecd3 bench: add `range_and_generate_series` (#19428)
add 65a6bc4231 chore: use extend instead of manual loop in multi group by
(#19429)
add 2c3566ce85 doc: add example for cache factory (#19139)
add 1acaf7a9a2 chore(deps): bump taiki-e/install-action from 2.64.0 to
2.64.2 (#19399)
add 9fe9ec744b fix: spark crc32 custom nullability (#19271)
add 9a9e4dd5c9 Add recursive protection on planner's
`create_physical_expr` (#19299)
add d9d55cfc6d chore(deps): bump aws-config from 1.8.11 to 1.8.12 (#19453)
add edc693f996 chore(deps): bump log from 0.4.28 to 0.4.29 (#19452)
add c7f9fdf90c chore(deps): bump taiki-e/install-action from 2.64.2 to
2.65.1 (#19451)
add 8e95627d3e chore(deps): bump sphinx-reredirects from 1.0.0 to 1.1.0 in
/docs (#19455)
add 5fedb84232 chore(deps): bump insta from 1.44.3 to 1.45.0 (#19454)
add a886b9eb42 added support for negative scale for log decimal32/64 and
power (#19409)
add 1e591640dd Remove core dependency from ffi (#19422)
add bb9a4a7ea1 bench: increase in_list benchmark coverage (#19443)
add 48f5d0b721 fix: Fix skip aggregate test to cover regression (#19461)
add d0d93117b9 fix: [19450]Added flush for tokio file(substrait) write
(#19456)
add 258e18cf88 Use SortMergeJoinExec name consistently in physical plan
outputs (#19246)
add d844f8687a Add:arrow_metadata() UDF (#19435)
add 4a1f69f9bc Update date_bin to support Time32 and Time64 data types
(#19341)
add 72f1746169 feat: Add decimal support for round (#19384)
add e6faacbf58 Fix panic during spill to disk in clickbench query (#19421)
add 677c543aef Optimize memory footprint of view arrays from
`ScalarValue::to_array_of_size` (#19441)
add 33ac70dd6d minor: refactoring of some `ScalarValue` code (#19439)
add 0bd880931e fix: csv schema_infer_max_records set to 0 return null
datatype (#19432)
add 902d3b32be fix: Add custom nullability for Spark LIKE function (#19218)
add 67b526a620 Refactor Spark crc32 & sha1 to remove unnecessary scalar
argument check (#19466)
add 47ddd50353 Add link to arrow-rs ticket in comments (#19479)
add a405d3fe4a Support nested field access in `get_field` with multiple
path arguments (#19389)
add d2830b6bed chore(deps): bump taiki-e/install-action from 2.65.1 to
2.65.2 (#19474)
add 6ce237492d Improve plan_to_sql handling of empty projections with
dialect-specific SELECT list support (#19221)
add ef2c1a30eb examples: replace sql_dialect with custom_sql_parser
example (#19383)
add 03904e1b2e Replace custom merge operator with arrow-rs implementation
(#19424)
add ea2e22c743 Implement nested recursive CTEs (#18956)
add e586ff5328 fix: implement custom nullability for spark abs function
(#19395)
add 058bcb0014 fix: custom nullability for format_string (#19173) (#19190)
add 62740802f6 Update `to_unixtime` udf function to support a consistent
set of argument types (#19442)
add ed7af0b12a Add: PI upper/lower bound f16 constants to ScalarValue
(#19497)
add 8532731570 chore: enforce clippy::allow_attributes for datafusion-ffi
crate (#19480)
add ae35177df5 Add CI check to ensure examples are documented in README
(#19371)
add e5ca510dcf perf: Improve performance of `to_hex` (> 2x) (#19503)
add d20c5d68f7 fix : snapshot to the modern multiline format (#19517)
add 5b90ceef1c perf: improve performance of string repeat (#19502)
add 134be4ce51 chore(deps): bump taiki-e/install-action from 2.65.2 to
2.65.3 (#19499)
add d825e5f394 docs : clarify unused test utility (#19508)
add bb4e0eca22 perf: Optimize `starts_with` and `ends_with` for scalar
arguments (#19516)
add 85c696df43 Date / time / interval arithmetic improvements (#19460)
add 8246631bfa fix: Implement `reset_state` for `LazyMemoryExec` (#19362)
add 6ac7b898e7 Preserve ORDER BY in Unparser for projection -> order by
pattern (#19483)
add 9eddf473fa fix: CteWorkTable: properly apply TableProvider::scan
projection argument (#18993)
add 4960284541 fix: Median() integer overflow (#19509)
add 10db6b3712 Redesign the try_reverse_output to support more cases
(#19446)
add a95c7fc2c4 feat: fix matching for named parameters with non-lowercase
signatures (#19378)
add 83ed19235b refactor: Spark `ascii` signature away from `user_defined`
(#19513)
add 36df145e97 feat: Add per-expression evaluation timing metrics to
ProjectionExec (#19447)
add 3aa0ab78b5 Fix: SparkAscii nullability to depend on input nullability
(#19531)
add 1dbf9a6df9 chore(deps): bump tracing from 0.1.41 to 0.1.43 (#19543)
add c37db42aeb chore(deps): bump substrait from 0.62.0 to 0.62.2 (#19542)
add d7e5190f78 chore(deps): bump taiki-e/install-action from 2.65.3 to
2.65.6 (#19541)
add bd10f27443 minor: run all examples by default (#19506)
add 94709dc025 perf: improve performance of string replace (#19530)
add 43567b4683 perf: improve performance of levenshtein by reusing cache
buffer (#19532)
add 3f0b3425cb feat: Improve sort memory resilience (#19494)
add 7c50448f5a perf: improve performance of translate by reusing buffers
(#19533)
add 673d7c93a0 Refactor TopKHashTable to use HashTable API (#19464)
add f9cdfea7fe docs: Improve config tables' readability (#19522)
add 8ac500bf07 Revert Spark Elt nullability change (#19510)
add 8469aa1dcd minor: implement more arms for `get_data_types()` for
`NativeType` (#19449)
add 1d2b389599 perf: Optimize `contains` for scalar search arg (#19529)
add 13f38435a2 Introduce `TypeSignatureClass::Any` (#19485)
add a6fd5cc840 Upgrade hashbrown to 0.16 (#19554)
add a51e3a079e minor : add crypto function benchmark (#19539)
add 3420a2d4ad chore(deps): bump taiki-e/install-action from 2.65.6 to
2.65.8 (#19559)
add 34addca6ba bugfix: preserve schema metadata for record batch in FFI
(#19293)
add d13d89129b feat: Add DELETE/UPDATE hooks to TableProvider trait and to
MemTable implementation (#19142)
add 1704d1e740 refactor: extract the data generate out of aggregate_topk
benchmark (#19523)
add 9690f958e2 perf: improve performance of lpad/rpad by reusing buffers
(#19558)
add 56a2be17d0 perf: optimize regexp_count to avoid String allocation when
start position is provided (#19553)
add 4e45c19d10 Enables DefaultListFilesCache by default (#19366)
add f1e5c94f3a Compute Dynamic Filters only when a consumer supports them
(#19546)
add 1ce4b51a48 Various refactors to string functions (#19402)
add 27de50d055 fix: Reverse row selection should respect the row group
index (#19557)
add 79f67b8ef2 feat: implement partition_statistics for WindowAggExec
(#18534)
add b818f93416 perf: Improve performance of `md5` (#19568)
add fd263216c9 feat: integrate batch coalescer with async fn exec (#19342)
add 8959b3d11e feat: output statistics for constant columns in projections
(#19419)
add db7b8cc4db Implement `partition_statistics` API for
`NestedLoopJoinExec` (#19468)
add cd12d51039 Replace deprecated structopt with clap in
datafusion-benchmarks (#19492)
add 818706ab78 feat: `to_time` function (#19540)
add 0db668bc99 Refactor duplicate code in `type_coercion/functions.rs`
(#19518)
add 90f5bfe309 feat: Implement Spark functions hour, minute, second
(#19512)
add bc753c201f chore(deps): bump taiki-e/install-action from 2.65.8 to
2.65.10 (#19578)
add 195d3d64bb perf: optimize strpos by eliminating double iteration for
UTF-8 (#19572)
add 9a9ff8d616 perf: Improve performance of hex encoding in spark
functions (#19586)
add 56fec71c7b Add left function benchmark (#19600)
add 987b94ca47 chore: Add TPCDS benchmark comparison for PR (#19552)
add 1320069246 Fix typo in contributor guide architecture section (#19613)
add a295698592 chore(deps): bump taiki-e/install-action from 2.65.10 to
2.65.11 (#19601)
add 715962c80b perf: optimize factorial function performance (#19575)
add 8809dae280 perf: Improve performance of ltrim, rtrim, btrim (#19551)
add 7695d71082 chore(deps): bump sphinx from 8.2.3 to 9.0.4 in /docs
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (cc3630ec73)
\
N -- N -- N refs/heads/dependabot/pip/docs/sphinx-9.0.4
(7695d71082)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/audit.yml | 2 +-
.github/workflows/dev.yml | 14 +-
.github/workflows/rust.yml | 21 +-
Cargo.lock | 375 ++--
Cargo.toml | 10 +-
benchmarks/Cargo.toml | 2 +-
benchmarks/README.md | 72 +-
benchmarks/bench.sh | 180 +-
.../clickbench/queries/{ => sorted_data}/q0.sql | 3 +-
benchmarks/src/bin/dfbench.rs | 19 +-
benchmarks/src/bin/external_aggr.rs | 31 +-
benchmarks/src/bin/imdb.rs | 24 +-
benchmarks/src/bin/mem_profile.rs | 49 +-
benchmarks/src/cancellation.rs | 28 +-
benchmarks/src/clickbench.rs | 143 +-
benchmarks/src/h2o.rs | 30 +-
benchmarks/src/hj.rs | 14 +-
benchmarks/src/imdb/convert.rs | 14 +-
benchmarks/src/imdb/run.rs | 30 +-
benchmarks/src/lib.rs | 1 +
benchmarks/src/nlj.rs | 18 +-
benchmarks/src/smj.rs | 524 +++++
benchmarks/src/sort_tpch.rs | 22 +-
benchmarks/src/tpcds/run.rs | 32 +-
benchmarks/src/tpch/run.rs | 36 +-
benchmarks/src/util/options.rs | 20 +-
benchmarks/src/util/run.rs | 2 +-
ci/scripts/check_examples_docs.sh | 64 +
ci/scripts/doc_prettier_check.sh | 57 +
datafusion-cli/Cargo.toml | 4 +-
datafusion-cli/examples/cli-session-context.rs | 2 +-
datafusion-cli/src/catalog.rs | 50 +-
datafusion-cli/src/cli_context.rs | 2 +-
datafusion-cli/src/command.rs | 2 +-
datafusion-cli/src/exec.rs | 27 +-
datafusion-cli/src/functions.rs | 120 +-
datafusion-cli/src/helper.rs | 10 +-
datafusion-cli/src/highlighter.rs | 4 +-
datafusion-cli/src/main.rs | 130 +-
datafusion-cli/src/object_storage.rs | 51 +-
datafusion-cli/src/object_storage/instrumented.rs | 114 +-
datafusion-cli/src/print_format.rs | 66 +-
datafusion-cli/src/print_options.rs | 8 +-
datafusion-cli/tests/cli_integration.rs | 34 +-
...errides@explain_plan_environment_overrides.snap | 1 -
.../tests/snapshots/[email protected] | 2 +-
datafusion-cli/tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
datafusion-cli/tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/cli_quick_test@batch_size.snap | 2 +-
.../cli_quick_test@default_explain_plan.snap | 1 -
.../tests/snapshots/[email protected] | 2 +-
.../tests/snapshots/[email protected] | 2 +-
datafusion-examples/Cargo.toml | 8 +-
datafusion-examples/README.md | 229 +-
.../examples/builtin_functions/date_time.rs | 7 +-
.../examples/builtin_functions/function_factory.rs | 2 +-
.../examples/builtin_functions/main.rs | 64 +-
.../examples/builtin_functions/regexp.rs | 60 +-
.../examples/custom_data_source/csv_json_opener.rs | 2 +-
.../custom_data_source/custom_datasource.rs | 8 +-
.../custom_data_source/custom_file_casts.rs | 18 +-
.../custom_data_source/custom_file_format.rs | 6 +-
.../custom_data_source/default_column_values.rs | 147 +-
.../custom_data_source/file_stream_provider.rs | 6 +-
.../examples/custom_data_source/main.rs | 83 +-
datafusion-examples/examples/data_io/catalog.rs | 4 +-
.../examples/data_io/json_shredding.rs | 114 +-
datafusion-examples/examples/data_io/main.rs | 95 +-
.../examples/data_io/parquet_advanced_index.rs | 14 +-
.../examples/data_io/parquet_embedded_index.rs | 30 +-
.../examples/data_io/parquet_encrypted.rs | 4 +-
.../examples/data_io/parquet_exec_visitor.rs | 23 +-
.../examples/data_io/parquet_index.rs | 12 +-
.../examples/data_io/remote_catalog.rs | 2 +-
.../examples/dataframe/cache_factory.rs | 233 ++
.../examples/dataframe/dataframe.rs | 6 +-
datafusion-examples/examples/dataframe/main.rs | 68 +-
.../examples/execution_monitoring/main.rs | 75 +-
.../memory_pool_execution_plan.rs | 2 +-
.../examples/execution_monitoring/tracing.rs | 14 +-
.../external_dependency/dataframe_to_s3.rs | 2 +-
.../examples/external_dependency/main.rs | 56 +-
.../ffi/ffi_example_table_provider/src/lib.rs | 7 +-
.../examples/ffi/ffi_module_interface/Cargo.toml | 2 +-
.../examples/ffi/ffi_module_interface/src/lib.rs | 7 +-
.../examples/ffi/ffi_module_loader/Cargo.toml | 3 +-
.../examples/ffi/ffi_module_loader/src/main.rs | 13 +-
datafusion-examples/examples/flight/main.rs | 59 +-
datafusion-examples/examples/flight/server.rs | 2 +-
datafusion-examples/examples/flight/sql_server.rs | 4 +-
.../examples/proto/composed_extension_codec.rs | 2 +-
datafusion-examples/examples/proto/main.rs | 54 +-
.../examples/query_planning/analyzer_rule.rs | 4 +-
.../examples/query_planning/expr_api.rs | 31 +-
.../examples/query_planning/main.rs | 87 +-
.../examples/query_planning/optimizer_rule.rs | 2 +-
.../examples/query_planning/plan_to_sql.rs | 2 +-
.../examples/relation_planner/main.rs | 121 ++
.../examples/relation_planner/match_recognize.rs | 406 ++++
.../examples/relation_planner/pivot_unpivot.rs | 567 +++++
.../examples/relation_planner/table_sample.rs | 845 ++++++++
datafusion-examples/examples/sql_ops/analysis.rs | 2 +-
.../examples/sql_ops/custom_sql_parser.rs | 420 ++++
datafusion-examples/examples/sql_ops/dialect.rs | 135 --
datafusion-examples/examples/sql_ops/frontend.rs | 2 +-
datafusion-examples/examples/sql_ops/main.rs | 79 +-
datafusion-examples/examples/sql_ops/query.rs | 10 +-
datafusion-examples/examples/udf/advanced_udaf.rs | 4 +-
datafusion-examples/examples/udf/advanced_udf.rs | 6 +-
datafusion-examples/examples/udf/async_udf.rs | 26 +-
datafusion-examples/examples/udf/main.rs | 87 +-
datafusion-examples/examples/udf/simple_udtf.rs | 9 +-
datafusion/catalog-listing/src/config.rs | 88 +-
datafusion/catalog-listing/src/helpers.rs | 21 +-
datafusion/catalog-listing/src/mod.rs | 3 +-
datafusion/catalog-listing/src/options.rs | 4 +-
datafusion/catalog-listing/src/table.rs | 218 +-
datafusion/catalog/src/async.rs | 10 +-
datafusion/catalog/src/catalog.rs | 2 +-
datafusion/catalog/src/cte_worktable.rs | 38 +-
datafusion/catalog/src/default_table_source.rs | 2 +-
datafusion/catalog/src/information_schema.rs | 20 +-
datafusion/catalog/src/lib.rs | 2 +-
datafusion/catalog/src/listing_schema.rs | 2 +-
datafusion/catalog/src/memory/schema.rs | 2 +-
datafusion/catalog/src/memory/table.rs | 351 ++-
datafusion/catalog/src/schema.rs | 2 +-
datafusion/catalog/src/stream.rs | 2 +-
datafusion/catalog/src/streaming.rs | 6 +-
datafusion/catalog/src/table.rs | 27 +-
datafusion/catalog/src/view.rs | 2 +-
datafusion/common-runtime/Cargo.toml | 2 +-
datafusion/common/Cargo.toml | 7 +-
datafusion/common/benches/with_hashes.rs | 209 ++
datafusion/common/src/config.rs | 111 +
datafusion/common/src/dfschema.rs | 2 +-
datafusion/common/src/file_options/csv_writer.rs | 17 +
datafusion/common/src/file_options/json_writer.rs | 18 +-
datafusion/common/src/hash_utils.rs | 136 +-
datafusion/common/src/lib.rs | 2 +-
datafusion/common/src/pruning.rs | 13 +
datafusion/common/src/scalar/consts.rs | 12 +
datafusion/common/src/scalar/mod.rs | 231 +-
datafusion/common/src/stats.rs | 243 ++-
datafusion/common/src/types/builtin.rs | 15 +
datafusion/common/src/utils/memory.rs | 200 +-
datafusion/common/src/utils/mod.rs | 36 +-
datafusion/core/Cargo.toml | 12 +-
datafusion/core/benches/data_utils/mod.rs | 2 +-
datafusion/core/benches/distinct_query_sql.rs | 4 +-
datafusion/core/benches/filter_query_sql.rs | 2 +-
datafusion/core/benches/map_query_sql.rs | 4 +-
datafusion/core/benches/parquet_query_sql.rs | 4 +-
datafusion/core/benches/physical_plan.rs | 2 +-
.../core/benches/preserve_file_partitioning.rs | 838 ++++++++
datafusion/core/benches/push_down_filter.rs | 12 +-
.../core/benches/range_and_generate_series.rs | 90 +
datafusion/core/benches/scalar.rs | 2 +-
datafusion/core/benches/sort.rs | 66 +-
datafusion/core/benches/sort_limit_query_sql.rs | 3 +-
datafusion/core/benches/spm.rs | 6 +-
datafusion/core/benches/sql_planner.rs | 10 +-
datafusion/core/benches/sql_planner_extended.rs | 2 +-
datafusion/core/benches/sql_query_with_io.rs | 6 +-
datafusion/core/benches/struct_query_sql.rs | 2 +-
datafusion/core/benches/topk_aggregate.rs | 126 +-
datafusion/core/src/bin/print_functions_docs.rs | 10 +-
datafusion/core/src/dataframe/mod.rs | 21 +-
datafusion/core/src/datasource/dynamic_file.rs | 2 +-
datafusion/core/src/datasource/empty.rs | 2 +-
datafusion/core/src/datasource/file_format/avro.rs | 40 +-
datafusion/core/src/datasource/file_format/csv.rs | 187 +-
datafusion/core/src/datasource/file_format/json.rs | 68 +-
datafusion/core/src/datasource/file_format/mod.rs | 29 +-
.../core/src/datasource/file_format/options.rs | 2 +-
.../core/src/datasource/file_format/parquet.rs | 43 +-
datafusion/core/src/datasource/listing/table.rs | 441 +---
.../core/src/datasource/listing_table_factory.rs | 6 +-
datafusion/core/src/datasource/memory_test.rs | 11 +-
datafusion/core/src/datasource/mod.rs | 83 +-
.../core/src/datasource/physical_plan/avro.rs | 90 +-
.../core/src/datasource/physical_plan/csv.rs | 164 +-
.../core/src/datasource/physical_plan/json.rs | 29 +-
.../core/src/datasource/physical_plan/mod.rs | 139 +-
.../core/src/datasource/physical_plan/parquet.rs | 256 +--
datafusion/core/src/datasource/view_test.rs | 36 +-
datafusion/core/src/execution/context/mod.rs | 169 +-
datafusion/core/src/execution/context/parquet.rs | 6 +-
datafusion/core/src/execution/session_state.rs | 138 +-
.../core/src/execution/session_state_defaults.rs | 6 +-
datafusion/core/src/lib.rs | 3 +
datafusion/core/src/physical_planner.rs | 361 +++-
datafusion/core/src/prelude.rs | 2 +-
datafusion/core/src/test/mod.rs | 12 +-
datafusion/core/src/test/object_store.rs | 10 +-
datafusion/core/src/test_util/parquet.rs | 13 +-
datafusion/core/tests/catalog/memory.rs | 10 +-
.../tests/catalog_listing/pruned_partition_list.rs | 4 +-
datafusion/core/tests/config_from_env.rs | 60 +-
datafusion/core/tests/core_integration.rs | 3 -
.../tests/custom_sources_cases/dml_planning.rs | 297 +++
datafusion/core/tests/custom_sources_cases/mod.rs | 9 +-
.../provider_filter_pushdown.rs | 14 +-
.../core/tests/custom_sources_cases/statistics.rs | 2 +
.../core/tests/dataframe/dataframe_functions.rs | 4 +-
datafusion/core/tests/dataframe/describe.rs | 2 +-
datafusion/core/tests/dataframe/mod.rs | 746 +++----
.../core/tests/datasource/object_store_access.rs | 76 +-
datafusion/core/tests/execution/coop.rs | 41 +-
.../core/tests/execution/datasource_split.rs | 2 +-
datafusion/core/tests/execution/logical_plan.rs | 2 +-
datafusion/core/tests/expr_api/mod.rs | 4 +-
datafusion/core/tests/expr_api/parse_sql_expr.rs | 2 +-
datafusion/core/tests/expr_api/simplification.rs | 22 +-
datafusion/core/tests/fifo/mod.rs | 8 +-
datafusion/core/tests/fuzz.rs | 5 +-
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 37 +-
.../aggregation_fuzzer/context_generator.rs | 2 +-
.../aggregation_fuzzer/data_generator.rs | 6 +-
.../tests/fuzz_cases/aggregation_fuzzer/fuzzer.rs | 4 +-
.../fuzz_cases/aggregation_fuzzer/query_builder.rs | 2 +-
.../tests/fuzz_cases/distinct_count_string_fuzz.rs | 2 +-
.../core/tests/fuzz_cases/equivalence/ordering.rs | 10 +-
.../tests/fuzz_cases/equivalence/projection.rs | 8 +-
.../tests/fuzz_cases/equivalence/properties.rs | 6 +-
.../core/tests/fuzz_cases/equivalence/utils.rs | 8 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 24 +-
datafusion/core/tests/fuzz_cases/limit_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/merge_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/pruning.rs | 4 +-
.../tests/fuzz_cases/record_batch_generator.rs | 20 +-
datafusion/core/tests/fuzz_cases/sort_fuzz.rs | 4 +-
.../fuzz_cases/sort_preserving_repartition_fuzz.rs | 11 +-
.../core/tests/fuzz_cases/sort_query_fuzz.rs | 28 +-
.../spilling_fuzz_in_memory_constrained_env.rs | 40 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 53 +-
.../memory_limit/memory_limit_validation/utils.rs | 6 +-
datafusion/core/tests/memory_limit/mod.rs | 28 +-
.../tests/memory_limit/repartition_mem_limit.rs | 15 +-
datafusion/core/tests/optimizer/mod.rs | 6 +-
datafusion/core/tests/parquet/custom_reader.rs | 6 +-
datafusion/core/tests/parquet/encryption.rs | 4 +-
.../parquet/{schema_adapter.rs => expr_adapter.rs} | 173 +-
.../core/tests/parquet/external_access_plan.rs | 10 +-
datafusion/core/tests/parquet/file_statistics.rs | 16 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 6 +-
datafusion/core/tests/parquet/mod.rs | 29 +-
datafusion/core/tests/parquet/page_pruning.rs | 8 +-
datafusion/core/tests/parquet/schema_coercion.rs | 6 +-
datafusion/core/tests/parquet/utils.rs | 11 +-
.../physical_optimizer/aggregate_statistics.rs | 27 +-
.../combine_partial_final_agg.rs | 15 +-
.../physical_optimizer/enforce_distribution.rs | 884 ++++----
.../tests/physical_optimizer/enforce_sorting.rs | 74 +-
.../enforce_sorting_monotonicity.rs | 2 +-
.../physical_optimizer/filter_pushdown/mod.rs | 971 +++++++--
.../physical_optimizer/filter_pushdown/util.rs | 42 +-
.../tests/physical_optimizer/join_selection.rs | 70 +-
.../tests/physical_optimizer/limit_pushdown.rs | 195 +-
.../limited_distinct_aggregation.rs | 6 +-
datafusion/core/tests/physical_optimizer/mod.rs | 1 +
.../physical_optimizer/partition_statistics.rs | 493 ++++-
.../physical_optimizer/projection_pushdown.rs | 365 ++--
.../core/tests/physical_optimizer/pushdown_sort.rs | 1040 +++++++++
.../replace_with_order_preserving_variants.rs | 10 +-
.../tests/physical_optimizer/sanity_checker.rs | 21 +-
.../core/tests/physical_optimizer/test_utils.rs | 310 ++-
.../tests/physical_optimizer/window_optimize.rs | 4 +-
datafusion/core/tests/schema_adapter/mod.rs | 18 -
.../schema_adapter_integration_tests.rs | 764 -------
datafusion/core/tests/sql/aggregates/basic.rs | 8 +-
datafusion/core/tests/sql/aggregates/dict_nulls.rs | 24 +-
datafusion/core/tests/sql/aggregates/mod.rs | 12 +-
datafusion/core/tests/sql/explain_analyze.rs | 55 +-
datafusion/core/tests/sql/joins.rs | 79 +-
datafusion/core/tests/sql/mod.rs | 8 +-
datafusion/core/tests/sql/path_partition.rs | 38 +-
datafusion/core/tests/sql/runtime_config.rs | 92 +
datafusion/core/tests/sql/select.rs | 2 +-
datafusion/core/tests/sql/unparser.rs | 462 ++++
datafusion/core/tests/tracing/asserting_tracer.rs | 2 +-
.../core/tests/tracing/traceable_object_store.rs | 4 +-
datafusion/core/tests/user_defined/expr_planner.rs | 18 +-
.../core/tests/user_defined/insert_operation.rs | 4 +-
datafusion/core/tests/user_defined/mod.rs | 3 +
.../core/tests/user_defined/relation_planner.rs | 527 +++++
.../tests/user_defined/user_defined_aggregates.rs | 80 +-
.../user_defined_async_scalar_functions.rs | 2 +-
.../core/tests/user_defined/user_defined_plan.rs | 73 +-
.../user_defined/user_defined_scalar_functions.rs | 69 +-
.../user_defined/user_defined_table_functions.rs | 16 +-
.../user_defined/user_defined_window_functions.rs | 303 +--
datafusion/datasource-arrow/src/file_format.rs | 33 +-
datafusion/datasource-arrow/src/source.rs | 126 +-
.../src/avro_to_arrow/arrow_array_reader.rs | 56 +-
.../datasource-avro/src/avro_to_arrow/schema.rs | 16 +-
datafusion/datasource-avro/src/file_format.rs | 4 +-
datafusion/datasource-avro/src/source.rs | 21 +-
datafusion/datasource-csv/Cargo.toml | 2 +-
datafusion/datasource-csv/src/file_format.rs | 67 +-
datafusion/datasource-csv/src/source.rs | 47 +-
datafusion/datasource-json/src/file_format.rs | 11 +-
datafusion/datasource-json/src/source.rs | 23 +-
datafusion/datasource-parquet/src/access_plan.rs | 18 +-
datafusion/datasource-parquet/src/file_format.rs | 19 +-
datafusion/datasource-parquet/src/metadata.rs | 207 +-
datafusion/datasource-parquet/src/mod.rs | 1 +
datafusion/datasource-parquet/src/opener.rs | 1261 ++++++++---
datafusion/datasource-parquet/src/page_filter.rs | 18 +-
datafusion/datasource-parquet/src/reader.rs | 4 +-
datafusion/datasource-parquet/src/row_filter.rs | 203 +-
.../datasource-parquet/src/row_group_filter.rs | 11 +-
datafusion/datasource-parquet/src/sort.rs | 1022 +++++++++
datafusion/datasource-parquet/src/source.rs | 243 ++-
datafusion/datasource/Cargo.toml | 2 +-
.../benches/split_groups_by_statistics.rs | 2 +-
datafusion/datasource/src/decoder.rs | 28 +-
datafusion/datasource/src/display.rs | 2 +-
datafusion/datasource/src/file.rs | 106 +-
datafusion/datasource/src/file_compression_type.rs | 60 +-
datafusion/datasource/src/file_format.rs | 2 +-
datafusion/datasource/src/file_groups.rs | 136 +-
datafusion/datasource/src/file_scan_config.rs | 401 +++-
datafusion/datasource/src/file_sink_config.rs | 4 +-
datafusion/datasource/src/file_stream.rs | 21 +-
datafusion/datasource/src/memory.rs | 22 +-
datafusion/datasource/src/mod.rs | 128 +-
datafusion/datasource/src/projection.rs | 17 +-
datafusion/datasource/src/schema_adapter.rs | 1064 +--------
datafusion/datasource/src/sink.rs | 6 +-
datafusion/datasource/src/source.rs | 35 +-
datafusion/datasource/src/statistics.rs | 6 +-
datafusion/datasource/src/table_schema.rs | 107 +-
datafusion/datasource/src/test_util.rs | 20 +-
datafusion/datasource/src/url.rs | 287 ++-
datafusion/datasource/src/write/demux.rs | 16 +-
datafusion/datasource/src/write/mod.rs | 25 +-
datafusion/datasource/src/write/orchestration.rs | 24 +-
datafusion/doc/Cargo.toml | 2 +-
datafusion/doc/src/lib.rs | 1 +
datafusion/execution/Cargo.toml | 4 +-
datafusion/execution/src/cache/cache_manager.rs | 159 +-
datafusion/execution/src/cache/cache_unit.rs | 749 +------
.../{cache_unit.rs => file_metadata_cache.rs} | 158 +-
datafusion/execution/src/cache/list_files_cache.rs | 427 +++-
datafusion/execution/src/cache/mod.rs | 2 +
datafusion/execution/src/config.rs | 80 +-
datafusion/execution/src/disk_manager.rs | 7 +-
datafusion/execution/src/lib.rs | 1 +
datafusion/execution/src/memory_pool/pool.rs | 34 +-
datafusion/execution/src/object_store.rs | 2 +-
datafusion/execution/src/runtime_env.rs | 48 +-
datafusion/expr-common/Cargo.toml | 2 +-
datafusion/expr-common/src/dyn_eq.rs | 4 +-
datafusion/expr-common/src/lib.rs | 1 +
datafusion/expr-common/src/signature.rs | 76 +-
datafusion/expr-common/src/type_coercion/binary.rs | 103 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 8 +-
.../src/type_coercion/binary/tests/comparison.rs | 58 +-
datafusion/expr/Cargo.toml | 2 +-
datafusion/expr/src/arguments.rs | 433 +++-
datafusion/expr/src/expr.rs | 8 +-
datafusion/expr/src/expr_rewriter/mod.rs | 6 +-
datafusion/expr/src/expr_schema.rs | 55 +-
datafusion/expr/src/lib.rs | 5 +
datafusion/expr/src/logical_plan/dml.rs | 8 +-
datafusion/expr/src/logical_plan/plan.rs | 1 +
datafusion/expr/src/planner.rs | 103 +-
datafusion/expr/src/ptr_eq.rs | 2 +-
datafusion/expr/src/type_coercion/functions.rs | 304 ++-
datafusion/expr/src/udf.rs | 12 +-
datafusion/expr/src/udf_eq.rs | 2 +-
datafusion/ffi/Cargo.toml | 23 +-
datafusion/ffi/src/arrow_wrappers.rs | 18 +-
datafusion/ffi/src/catalog_provider.rs | 238 +-
datafusion/ffi/src/catalog_provider_list.rs | 196 +-
datafusion/ffi/src/execution/task_ctx.rs | 85 +-
datafusion/ffi/src/execution/task_ctx_provider.rs | 57 +-
datafusion/ffi/src/execution_plan.rs | 162 +-
datafusion/ffi/src/expr/columnar_value.rs | 1 -
datafusion/ffi/src/expr/distribution.rs | 11 +-
datafusion/ffi/src/expr/expr_properties.rs | 3 -
datafusion/ffi/src/expr/interval.rs | 4 +-
datafusion/ffi/src/insert_op.rs | 3 +-
datafusion/ffi/src/lib.rs | 4 +-
datafusion/ffi/src/physical_expr/mod.rs | 302 +--
datafusion/ffi/src/physical_expr/partitioning.rs | 7 +-
datafusion/ffi/src/physical_expr/sort.rs | 7 +-
datafusion/ffi/src/plan_properties.rs | 155 +-
.../ffi/src/proto/logical_extension_codec.rs | 723 +++++++
.../tests/catalog_listing => ffi/src/proto}/mod.rs | 3 +-
.../ffi/src/proto/physical_extension_codec.rs | 677 ++++++
datafusion/ffi/src/record_batch_stream.rs | 125 +-
datafusion/ffi/src/schema_provider.rs | 246 ++-
.../src/{session_config.rs => session/config.rs} | 35 +-
datafusion/ffi/src/session/mod.rs | 621 ++++++
datafusion/ffi/src/table_provider.rs | 292 +--
datafusion/ffi/src/table_source.rs | 7 +-
datafusion/ffi/src/tests/async_provider.rs | 64 +-
datafusion/ffi/src/tests/catalog.rs | 49 +-
datafusion/ffi/src/tests/mod.rs | 61 +-
datafusion/ffi/src/tests/sync_provider.rs | 11 +-
datafusion/ffi/src/tests/udf_udaf_udwf.rs | 70 +-
datafusion/ffi/src/tests/utils.rs | 8 +-
datafusion/ffi/src/udaf/accumulator.rs | 156 +-
datafusion/ffi/src/udaf/accumulator_args.rs | 122 +-
datafusion/ffi/src/udaf/groups_accumulator.rs | 176 +-
datafusion/ffi/src/udaf/mod.rs | 288 +--
datafusion/ffi/src/udf/mod.rs | 168 +-
datafusion/ffi/src/udf/return_type_args.rs | 16 +-
datafusion/ffi/src/udtf.rs | 169 +-
datafusion/ffi/src/udwf/mod.rs | 170 +-
datafusion/ffi/src/udwf/partition_evaluator.rs | 150 +-
.../ffi/src/udwf/partition_evaluator_args.rs | 108 +-
datafusion/ffi/src/udwf/range.rs | 1 -
datafusion/ffi/src/util.rs | 37 +-
datafusion/ffi/src/volatility.rs | 3 +-
datafusion/ffi/tests/ffi_catalog.rs | 14 +-
datafusion/ffi/tests/ffi_integration.rs | 11 +-
datafusion/ffi/tests/ffi_udaf.rs | 73 +-
datafusion/ffi/tests/ffi_udf.rs | 6 +-
datafusion/ffi/tests/ffi_udtf.rs | 10 +-
datafusion/ffi/tests/ffi_udwf.rs | 7 +-
datafusion/ffi/tests/utils/mod.rs | 43 +
datafusion/functions-aggregate-common/Cargo.toml | 2 +-
datafusion/functions-aggregate/Cargo.toml | 2 +-
datafusion/functions-aggregate/src/array_agg.rs | 2 +-
datafusion/functions-aggregate/src/first_last.rs | 33 +-
datafusion/functions-aggregate/src/median.rs | 76 +-
datafusion/functions-nested/benches/array_slice.rs | 4 +-
datafusion/functions-nested/benches/map.rs | 6 +-
datafusion/functions-nested/src/array_has.rs | 26 +-
datafusion/functions-nested/src/cardinality.rs | 4 +-
datafusion/functions-nested/src/concat.rs | 6 +-
datafusion/functions-nested/src/dimension.rs | 2 +-
datafusion/functions-nested/src/distance.rs | 4 +-
datafusion/functions-nested/src/empty.rs | 2 +-
datafusion/functions-nested/src/except.rs | 6 +-
datafusion/functions-nested/src/extract.rs | 14 +-
datafusion/functions-nested/src/flatten.rs | 2 +-
datafusion/functions-nested/src/length.rs | 2 +-
datafusion/functions-nested/src/make_array.rs | 8 +-
datafusion/functions-nested/src/map.rs | 2 +-
datafusion/functions-nested/src/map_entries.rs | 2 +-
datafusion/functions-nested/src/map_extract.rs | 4 +-
datafusion/functions-nested/src/map_keys.rs | 2 +-
datafusion/functions-nested/src/map_values.rs | 2 +-
datafusion/functions-nested/src/min_max.rs | 4 +-
datafusion/functions-nested/src/planner.rs | 9 +-
datafusion/functions-nested/src/position.rs | 6 +-
datafusion/functions-nested/src/range.rs | 16 +-
datafusion/functions-nested/src/remove.rs | 482 ++++-
datafusion/functions-nested/src/repeat.rs | 6 +-
datafusion/functions-nested/src/replace.rs | 6 +-
datafusion/functions-nested/src/resize.rs | 8 +-
datafusion/functions-nested/src/reverse.rs | 2 +-
datafusion/functions-nested/src/set_ops.rs | 14 +-
datafusion/functions-nested/src/sort.rs | 4 +-
datafusion/functions-nested/src/string.rs | 76 +-
datafusion/functions-nested/src/utils.rs | 2 +-
datafusion/functions-table/src/generate_series.rs | 87 +-
datafusion/functions-window-common/Cargo.toml | 2 +-
datafusion/functions-window/Cargo.toml | 2 +-
datafusion/functions/Cargo.toml | 56 +-
datafusion/functions/benches/contains.rs | 185 ++
.../functions/benches/{upper.rs => crypto.rs} | 49 +-
datafusion/functions/benches/ends_with.rs | 185 ++
.../functions/benches/{uuid.rs => factorial.rs} | 35 +-
datafusion/functions/benches/left.rs | 111 +
datafusion/functions/benches/levenshtein.rs | 87 +
datafusion/functions/benches/pad.rs | 314 ++-
datafusion/functions/benches/regexp_count.rs | 118 +
.../functions/benches/{repeat.rs => replace.rs} | 108 +-
datafusion/functions/benches/starts_with.rs | 185 ++
datafusion/functions/benches/to_hex.rs | 120 +-
datafusion/functions/benches/translate.rs | 90 +
datafusion/functions/benches/{ltrim.rs => trim.rs} | 200 +-
datafusion/functions/src/core/arrow_cast.rs | 13 +-
datafusion/functions/src/core/arrow_metadata.rs | 160 ++
datafusion/functions/src/core/getfield.rs | 645 ++++--
datafusion/functions/src/core/mod.rs | 14 +
datafusion/functions/src/crypto/basic.rs | 34 +-
datafusion/functions/src/datetime/date_bin.rs | 314 ++-
datafusion/functions/src/datetime/make_date.rs | 333 +--
datafusion/functions/src/datetime/make_time.rs | 267 +++
datafusion/functions/src/datetime/mod.rs | 16 +-
datafusion/functions/src/datetime/to_date.rs | 39 +-
datafusion/functions/src/datetime/to_time.rs | 252 +++
datafusion/functions/src/datetime/to_unixtime.rs | 38 +-
datafusion/functions/src/encoding/inner.rs | 728 ++++---
datafusion/functions/src/math/ceil.rs | 174 ++
datafusion/functions/src/math/decimal.rs | 111 +
datafusion/functions/src/math/factorial.rs | 78 +-
datafusion/functions/src/math/floor.rs | 174 ++
datafusion/functions/src/math/log.rs | 121 +-
datafusion/functions/src/math/mod.rs | 21 +-
datafusion/functions/src/math/monotonicity.rs | 48 -
datafusion/functions/src/math/power.rs | 424 +---
datafusion/functions/src/math/round.rs | 367 ++--
datafusion/functions/src/regex/regexpcount.rs | 12 +-
datafusion/functions/src/string/btrim.rs | 2 +-
datafusion/functions/src/string/common.rs | 234 +-
datafusion/functions/src/string/contains.rs | 89 +-
datafusion/functions/src/string/ends_with.rs | 294 ++-
datafusion/functions/src/string/levenshtein.rs | 24 +-
datafusion/functions/src/string/ltrim.rs | 10 +-
datafusion/functions/src/string/repeat.rs | 33 +-
datafusion/functions/src/string/replace.rs | 85 +-
datafusion/functions/src/string/rtrim.rs | 10 +-
datafusion/functions/src/string/split_part.rs | 24 +-
datafusion/functions/src/string/starts_with.rs | 265 ++-
datafusion/functions/src/string/to_hex.rs | 217 +-
datafusion/functions/src/string/uuid.rs | 2 +-
datafusion/functions/src/unicode/lpad.rs | 33 +-
datafusion/functions/src/unicode/rpad.rs | 38 +-
datafusion/functions/src/unicode/strpos.rs | 39 +-
datafusion/functions/src/unicode/translate.rs | 56 +-
datafusion/functions/src/utils.rs | 153 +-
datafusion/macros/Cargo.toml | 2 +-
datafusion/macros/src/user_doc.rs | 1 +
.../optimizer/benches/projection_unnecessary.rs | 4 +-
.../optimizer/src/analyzer/function_rewrite.rs | 2 +-
datafusion/optimizer/src/analyzer/mod.rs | 2 +-
.../src/analyzer/resolve_grouping_function.rs | 8 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 230 +-
.../optimizer/src/common_subexpr_eliminate.rs | 14 +-
datafusion/optimizer/src/decorrelate.rs | 6 +-
.../optimizer/src/decorrelate_lateral_join.rs | 6 +-
.../src/decorrelate_predicate_subquery.rs | 16 +-
datafusion/optimizer/src/eliminate_cross_join.rs | 19 +-
.../optimizer/src/eliminate_duplicated_expr.rs | 6 +-
datafusion/optimizer/src/eliminate_filter.rs | 6 +-
.../optimizer/src/eliminate_group_by_constant.rs | 8 +-
datafusion/optimizer/src/eliminate_join.rs | 4 +-
datafusion/optimizer/src/eliminate_limit.rs | 10 +-
datafusion/optimizer/src/eliminate_outer_join.rs | 6 +-
.../optimizer/src/extract_equijoin_predicate.rs | 6 +-
datafusion/optimizer/src/filter_null_join_keys.rs | 6 +-
datafusion/optimizer/src/join_key_set.rs | 2 +-
datafusion/optimizer/src/lib.rs | 1 +
.../optimizer/src/optimize_projections/mod.rs | 30 +-
datafusion/optimizer/src/optimize_unions.rs | 12 +-
datafusion/optimizer/src/optimizer.rs | 16 +-
datafusion/optimizer/src/plan_signature.rs | 2 +-
.../optimizer/src/propagate_empty_relation.rs | 18 +-
datafusion/optimizer/src/push_down_filter.rs | 24 +-
datafusion/optimizer/src/push_down_limit.rs | 16 +-
.../optimizer/src/replace_distinct_aggregate.rs | 6 +-
.../optimizer/src/scalar_subquery_to_join.rs | 50 +-
.../src/simplify_expressions/expr_simplifier.rs | 38 +-
.../src/simplify_expressions/inlist_simplifier.rs | 92 +-
.../optimizer/src/simplify_expressions/regex.rs | 20 +-
.../src/simplify_expressions/simplify_exprs.rs | 27 +-
.../src/simplify_expressions/unwrap_cast.rs | 6 +-
.../optimizer/src/simplify_expressions/utils.rs | 4 +-
.../optimizer/src/single_distinct_to_groupby.rs | 9 +-
datafusion/optimizer/src/test/mod.rs | 4 +-
datafusion/optimizer/src/test/user_defined.rs | 2 +-
datafusion/optimizer/src/utils.rs | 6 +-
.../optimizer/tests/optimizer_integration.rs | 371 ++--
datafusion/physical-expr-adapter/Cargo.toml | 2 +-
datafusion/physical-expr-adapter/src/lib.rs | 2 +-
.../physical-expr-adapter/src/schema_rewriter.rs | 214 +-
datafusion/physical-expr-common/Cargo.toml | 5 +-
datafusion/physical-expr-common/src/lib.rs | 1 +
.../src/metrics/baseline.rs | 10 +-
.../src/metrics/builder.rs | 4 +-
.../src/metrics/custom.rs | 2 +-
.../physical-expr-common/src/metrics/expression.rs | 88 +
.../src/metrics/mod.rs | 34 +-
.../src/metrics/value.rs | 12 +-
.../physical-expr-common/src/physical_expr.rs | 20 +-
datafusion/physical-expr-common/src/sort_expr.rs | 124 +-
datafusion/physical-expr-common/src/utils.rs | 17 +-
datafusion/physical-expr/Cargo.toml | 7 +-
datafusion/physical-expr/benches/in_list.rs | 278 ++-
datafusion/physical-expr/src/equivalence/class.rs | 8 +-
datafusion/physical-expr/src/expressions/case.rs | 991 ++++++---
.../literal_lookup_table/boolean_lookup_table.rs | 122 ++
.../bytes_like_lookup_table.rs | 223 ++
.../expressions/case/literal_lookup_table/mod.rs | 327 +++
.../literal_lookup_table/primitive_lookup_table.rs | 229 ++
.../src/expressions/dynamic_filters.rs | 150 +-
.../physical-expr/src/expressions/in_list.rs | 2030 ++++++++++++------
datafusion/physical-expr/src/partitioning.rs | 586 ++++-
datafusion/physical-expr/src/planner.rs | 33 +-
datafusion/physical-expr/src/projection.rs | 380 +++-
datafusion/physical-expr/src/scalar_function.rs | 8 +-
datafusion/physical-expr/src/simplifier/mod.rs | 127 +-
datafusion/physical-expr/src/utils/mod.rs | 2 +-
.../physical-optimizer/src/aggregate_statistics.rs | 57 +-
.../physical-optimizer/src/coalesce_batches.rs | 30 +-
.../src/combine_partial_final_agg.rs | 11 +-
.../physical-optimizer/src/enforce_distribution.rs | 334 +--
.../physical-optimizer/src/enforce_sorting/mod.rs | 10 +-
.../replace_with_order_preserving_variants.rs | 4 +-
.../src/enforce_sorting/sort_pushdown.rs | 6 +-
datafusion/physical-optimizer/src/ensure_coop.rs | 26 +-
.../physical-optimizer/src/filter_pushdown.rs | 21 +-
.../physical-optimizer/src/join_selection.rs | 33 +-
datafusion/physical-optimizer/src/lib.rs | 3 +-
.../physical-optimizer/src/limit_pushdown.rs | 7 +-
.../src/limit_pushdown_past_window.rs | 18 +-
.../src/limited_distinct_aggregation.rs | 28 +-
datafusion/physical-optimizer/src/optimizer.rs | 80 +-
.../physical-optimizer/src/output_requirements.rs | 9 +-
.../physical-optimizer/src/projection_pushdown.rs | 22 +-
datafusion/physical-optimizer/src/pushdown_sort.rs | 129 ++
.../physical-optimizer/src/sanity_checker.rs | 27 +-
.../physical-optimizer/src/topk_aggregation.rs | 12 +-
.../physical-optimizer/src/update_aggr_exprs.rs | 13 +-
datafusion/physical-plan/Cargo.toml | 2 +-
.../physical-plan/benches/aggregate_vectorized.rs | 4 +-
.../physical-plan/benches/partial_ordering.rs | 2 +-
.../physical-plan/benches/sort_preserving_merge.rs | 4 +-
datafusion/physical-plan/benches/spill_io.rs | 4 +-
.../src/aggregates/group_values/metrics.rs | 2 +-
.../src/aggregates/group_values/mod.rs | 4 +-
.../group_values/multi_group_by/boolean.rs | 2 +-
.../group_values/multi_group_by/bytes.rs | 10 +-
.../group_values/multi_group_by/bytes_view.rs | 163 +-
.../aggregates/group_values/multi_group_by/mod.rs | 38 +-
.../group_values/multi_group_by/primitive.rs | 4 +-
.../src/aggregates/group_values/row.rs | 11 +-
.../group_values/single_group_by/boolean.rs | 3 +-
.../group_values/single_group_by/bytes.rs | 4 +-
.../group_values/single_group_by/bytes_view.rs | 4 +-
.../group_values/single_group_by/primitive.rs | 14 +-
datafusion/physical-plan/src/aggregates/mod.rs | 633 ++++--
.../physical-plan/src/aggregates/no_grouping.rs | 32 +-
.../physical-plan/src/aggregates/row_hash.rs | 387 ++--
.../src/aggregates/topk/hash_table.rs | 200 +-
.../physical-plan/src/aggregates/topk/heap.rs | 135 +-
.../src/aggregates/topk/priority_map.rs | 172 +-
.../physical-plan/src/aggregates/topk_stream.rs | 8 +-
datafusion/physical-plan/src/analyze.rs | 4 +-
datafusion/physical-plan/src/async_func.rs | 135 +-
datafusion/physical-plan/src/coalesce/mod.rs | 2 +-
datafusion/physical-plan/src/coalesce_batches.rs | 29 +
.../physical-plan/src/coalesce_partitions.rs | 44 +-
datafusion/physical-plan/src/common.rs | 5 +-
datafusion/physical-plan/src/coop.rs | 43 +-
datafusion/physical-plan/src/display.rs | 4 +-
datafusion/physical-plan/src/empty.rs | 13 +-
datafusion/physical-plan/src/execution_plan.rs | 38 +-
datafusion/physical-plan/src/explain.rs | 15 +-
datafusion/physical-plan/src/filter.rs | 59 +-
datafusion/physical-plan/src/joins/cross_join.rs | 62 +-
.../physical-plan/src/joins/hash_join/exec.rs | 333 +--
.../src/joins/hash_join/inlist_builder.rs | 133 ++
.../physical-plan/src/joins/hash_join/mod.rs | 2 +
.../src/joins/hash_join/partitioned_hash_eval.rs | 493 ++++-
.../src/joins/hash_join/shared_bounds.rs | 409 +++-
.../physical-plan/src/joins/hash_join/stream.rs | 55 +-
.../physical-plan/src/joins/join_hash_map.rs | 19 +-
datafusion/physical-plan/src/joins/mod.rs | 8 +-
.../physical-plan/src/joins/nested_loop_join.rs | 325 +--
.../src/joins/piecewise_merge_join/classic_join.rs | 138 +-
.../src/joins/piecewise_merge_join/exec.rs | 21 +-
.../src/joins/sort_merge_join/exec.rs | 23 +-
.../src/joins/sort_merge_join/stream.rs | 697 ++++--
.../src/joins/sort_merge_join/tests.rs | 768 +++----
.../physical-plan/src/joins/stream_join_utils.rs | 110 +-
.../physical-plan/src/joins/symmetric_hash_join.rs | 28 +-
datafusion/physical-plan/src/joins/test_utils.rs | 6 +-
datafusion/physical-plan/src/joins/utils.rs | 134 +-
datafusion/physical-plan/src/lib.rs | 16 +-
datafusion/physical-plan/src/limit.rs | 11 +-
datafusion/physical-plan/src/memory.rs | 58 +-
.../src/metrics.rs} | 8 +-
datafusion/physical-plan/src/placeholder_row.rs | 13 +-
datafusion/physical-plan/src/projection.rs | 94 +-
datafusion/physical-plan/src/recursive_query.rs | 12 +-
.../src/repartition/distributor_channels.rs | 4 +-
datafusion/physical-plan/src/repartition/mod.rs | 190 +-
datafusion/physical-plan/src/sort_pushdown.rs | 120 ++
datafusion/physical-plan/src/sorts/cursor.rs | 4 +-
datafusion/physical-plan/src/sorts/merge.rs | 4 +-
.../physical-plan/src/sorts/multi_level_merge.rs | 18 +-
datafusion/physical-plan/src/sorts/partial_sort.rs | 159 +-
datafusion/physical-plan/src/sorts/sort.rs | 561 ++++-
.../src/sorts/sort_preserving_merge.rs | 172 +-
datafusion/physical-plan/src/sorts/stream.rs | 6 +-
.../physical-plan/src/sorts/streaming_merge.rs | 2 +-
.../src/spill/in_progress_spill_file.rs | 2 +-
datafusion/physical-plan/src/spill/mod.rs | 214 +-
.../physical-plan/src/spill/spill_manager.rs | 9 +-
datafusion/physical-plan/src/spill/spill_pool.rs | 67 +-
datafusion/physical-plan/src/stream.rs | 191 +-
datafusion/physical-plan/src/streaming.rs | 8 +-
datafusion/physical-plan/src/test.rs | 13 +-
datafusion/physical-plan/src/test/exec.rs | 12 +-
datafusion/physical-plan/src/topk/mod.rs | 18 +-
datafusion/physical-plan/src/tree_node.rs | 4 +-
datafusion/physical-plan/src/union.rs | 62 +-
datafusion/physical-plan/src/unnest.rs | 64 +-
.../src/windows/bounded_window_agg_exec.rs | 130 +-
datafusion/physical-plan/src/windows/mod.rs | 56 +-
.../physical-plan/src/windows/window_agg_exec.rs | 42 +-
datafusion/physical-plan/src/work_table.rs | 107 +-
datafusion/proto-common/Cargo.toml | 2 +-
.../proto-common/proto/datafusion_common.proto | 5 +
datafusion/proto-common/src/from_proto/mod.rs | 9 +-
datafusion/proto-common/src/generated/mod.rs | 1 +
datafusion/proto-common/src/generated/pbjson.rs | 58 +
datafusion/proto-common/src/generated/prost.rs | 8 +
datafusion/proto-common/src/lib.rs | 1 +
datafusion/proto-common/src/to_proto/mod.rs | 3 +
datafusion/proto/Cargo.toml | 1 +
datafusion/proto/proto/datafusion.proto | 19 +
datafusion/proto/src/bytes/mod.rs | 8 +-
datafusion/proto/src/bytes/registry.rs | 29 +-
datafusion/proto/src/common.rs | 2 +-
.../proto/src/generated/datafusion_proto_common.rs | 8 +
datafusion/proto/src/generated/mod.rs | 2 +
datafusion/proto/src/generated/pbjson.rs | 366 ++++
datafusion/proto/src/generated/prost.rs | 34 +-
datafusion/proto/src/lib.rs | 1 +
datafusion/proto/src/logical_plan/file_formats.rs | 21 +-
datafusion/proto/src/logical_plan/from_proto.rs | 15 +-
datafusion/proto/src/logical_plan/mod.rs | 48 +-
datafusion/proto/src/logical_plan/to_proto.rs | 45 +-
datafusion/proto/src/physical_plan/from_proto.rs | 25 +-
datafusion/proto/src/physical_plan/mod.rs | 136 +-
datafusion/proto/src/physical_plan/to_proto.rs | 47 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 38 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 239 ++-
datafusion/proto/tests/cases/serialize.rs | 6 +-
datafusion/pruning/src/file_pruner.rs | 131 +-
datafusion/pruning/src/lib.rs | 5 +-
datafusion/pruning/src/pruning_predicate.rs | 356 ++-
datafusion/session/src/lib.rs | 1 +
datafusion/session/src/session.rs | 2 +-
datafusion/spark/benches/char.rs | 2 +-
datafusion/spark/src/function/aggregate/avg.rs | 6 +-
datafusion/spark/src/function/aggregate/mod.rs | 12 +-
datafusion/spark/src/function/aggregate/try_sum.rs | 660 ++++++
datafusion/spark/src/function/array/shuffle.rs | 4 +-
datafusion/spark/src/function/array/spark_array.rs | 4 +-
.../spark/src/function/bitmap/bitmap_count.rs | 73 +-
datafusion/spark/src/function/bitwise/bit_count.rs | 60 +-
datafusion/spark/src/function/bitwise/bit_get.rs | 62 +-
datafusion/spark/src/function/bitwise/bit_shift.rs | 78 +-
.../spark/src/function/bitwise/bitwise_not.rs | 130 +-
datafusion/spark/src/function/conditional/if.rs | 6 +-
datafusion/spark/src/function/datetime/date_add.rs | 92 +-
datafusion/spark/src/function/datetime/date_sub.rs | 84 +-
datafusion/spark/src/function/datetime/extract.rs | 268 +++
datafusion/spark/src/function/datetime/last_day.rs | 82 +-
.../src/function/datetime/make_dt_interval.rs | 95 +-
.../spark/src/function/datetime/make_interval.rs | 6 +-
datafusion/spark/src/function/datetime/mod.rs | 18 +
datafusion/spark/src/function/datetime/next_day.rs | 89 +-
datafusion/spark/src/function/error_utils.rs | 6 +-
datafusion/spark/src/function/hash/crc32.rs | 47 +-
datafusion/spark/src/function/hash/sha1.rs | 58 +-
datafusion/spark/src/function/hash/sha2.rs | 35 +-
.../spark/src/function/map/map_from_arrays.rs | 88 +-
.../spark/src/function/map/map_from_entries.rs | 103 +-
datafusion/spark/src/function/map/utils.rs | 11 +-
datafusion/spark/src/function/math/abs.rs | 80 +-
datafusion/spark/src/function/math/factorial.rs | 4 +-
datafusion/spark/src/function/math/hex.rs | 126 +-
datafusion/spark/src/function/math/mod.rs | 6 +-
datafusion/spark/src/function/math/modulus.rs | 10 +-
datafusion/spark/src/function/math/rint.rs | 4 +-
datafusion/spark/src/function/math/width_bucket.rs | 149 +-
datafusion/spark/src/function/string/ascii.rs | 77 +-
datafusion/spark/src/function/string/char.rs | 63 +-
datafusion/spark/src/function/string/concat.rs | 77 +-
datafusion/spark/src/function/string/elt.rs | 2 +-
.../spark/src/function/string/format_string.rs | 81 +-
datafusion/spark/src/function/string/ilike.rs | 90 +-
datafusion/spark/src/function/string/like.rs | 92 +-
datafusion/spark/src/function/string/luhn_check.rs | 2 +-
datafusion/spark/src/function/url/mod.rs | 18 +-
datafusion/spark/src/function/url/parse_url.rs | 11 +-
datafusion/spark/src/function/url/try_parse_url.rs | 2 +-
.../spark/src/function/url/try_url_decode.rs | 4 +-
datafusion/spark/src/function/url/url_decode.rs | 6 +-
datafusion/spark/src/function/url/url_encode.rs | 2 +-
datafusion/spark/src/lib.rs | 5 +-
datafusion/sql/examples/sql.rs | 6 +-
datafusion/sql/src/cte.rs | 13 +-
datafusion/sql/src/expr/binary_op.rs | 2 +-
datafusion/sql/src/expr/function.rs | 74 +-
datafusion/sql/src/expr/identifier.rs | 79 +-
datafusion/sql/src/expr/mod.rs | 26 +-
datafusion/sql/src/expr/order_by.rs | 2 +-
datafusion/sql/src/expr/subquery.rs | 18 +-
datafusion/sql/src/expr/substring.rs | 10 +-
datafusion/sql/src/expr/unary_op.rs | 4 +-
datafusion/sql/src/expr/value.rs | 165 +-
datafusion/sql/src/lib.rs | 1 +
datafusion/sql/src/parser.rs | 65 +-
datafusion/sql/src/planner.rs | 22 +-
datafusion/sql/src/query.rs | 2 +-
datafusion/sql/src/relation/join.rs | 2 +-
datafusion/sql/src/relation/mod.rs | 123 +-
datafusion/sql/src/select.rs | 88 +-
datafusion/sql/src/set_expr.rs | 2 +-
datafusion/sql/src/statement.rs | 70 +-
datafusion/sql/src/unparser/ast.rs | 66 +-
datafusion/sql/src/unparser/dialect.rs | 49 +-
datafusion/sql/src/unparser/expr.rs | 91 +-
datafusion/sql/src/unparser/extension_unparser.rs | 4 +-
datafusion/sql/src/unparser/plan.rs | 187 +-
datafusion/sql/src/unparser/rewrite.rs | 13 +-
datafusion/sql/src/unparser/utils.rs | 48 +-
datafusion/sql/src/utils.rs | 45 +-
datafusion/sql/tests/cases/collection.rs | 10 +-
datafusion/sql/tests/cases/diagnostic.rs | 3 +-
datafusion/sql/tests/cases/params.rs | 87 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 272 ++-
datafusion/sql/tests/common/mod.rs | 2 +-
datafusion/sql/tests/sql_integration.rs | 2262 ++++++++++----------
datafusion/sqllogictest/Cargo.toml | 2 +-
datafusion/sqllogictest/bin/postgres_container.rs | 10 +-
datafusion/sqllogictest/bin/sqllogictests.rs | 40 +-
datafusion/sqllogictest/src/engines/conversion.rs | 4 +-
.../src/engines/datafusion_engine/runner.rs | 2 +-
.../runner.rs | 2 +-
datafusion/sqllogictest/src/engines/mod.rs | 4 +-
.../src/engines/postgres_engine/mod.rs | 4 +-
datafusion/sqllogictest/src/filters.rs | 19 +-
datafusion/sqllogictest/src/lib.rs | 4 +-
datafusion/sqllogictest/src/test_context.rs | 8 +-
datafusion/sqllogictest/src/util.rs | 2 +-
.../test_files/agg_func_substitute.slt | 27 +-
datafusion/sqllogictest/test_files/aggregate.slt | 223 +-
.../test_files/aggregate_repartition.slt | 16 +-
.../test_files/aggregate_skip_partial.slt | 17 +-
.../sqllogictest/test_files/aggregates_topk.slt | 49 +-
.../sqllogictest/test_files/arrow_typeof.slt | 5 +-
datafusion/sqllogictest/test_files/async_udf.slt | 15 +-
datafusion/sqllogictest/test_files/cast.slt | 36 +
.../sqllogictest/test_files/count_star_rule.slt | 14 +-
.../test_files/create_external_table.slt | 2 +-
datafusion/sqllogictest/test_files/cte.slt | 79 +-
.../test_files/datetime/arith_date_date.slt | 16 +
.../test_files/datetime/arith_date_integer.slt | 89 +
.../test_files/datetime/arith_date_interval.slt | 37 +
.../test_files/datetime/arith_date_time.slt | 116 +
.../test_files/datetime/arith_interval_double.slt | 41 +
.../datetime/arith_interval_interval.slt | 27 +
.../test_files/datetime/arith_negate_interval.slt | 13 +
.../test_files/datetime/arith_time_interval.slt | 70 +
.../test_files/datetime/arith_time_time.slt | 47 +
.../datetime/arith_timestamp_duration.slt | 147 ++
.../datetime/arith_timestamp_interval.slt | 36 +
.../datetime/arith_timestamp_timestamp.slt | 13 +
.../{ => datetime}/current_date_timezone.slt | 0
.../{ => datetime}/current_time_timezone.slt | 0
.../test_files/{expr => datetime}/date_part.slt | 0
.../test_files/{ => datetime}/dates.slt | 102 +
.../test_files/{ => datetime}/interval.slt | 0
.../test_files/{ => datetime}/interval_mysql.slt | 0
.../test_files/{ => datetime}/timestamps.slt | 676 +++++-
datafusion/sqllogictest/test_files/decimal.slt | 167 +-
datafusion/sqllogictest/test_files/delete.slt | 16 +-
datafusion/sqllogictest/test_files/distinct_on.slt | 9 +-
datafusion/sqllogictest/test_files/dml_delete.slt | 202 ++
datafusion/sqllogictest/test_files/dml_update.slt | 286 +++
datafusion/sqllogictest/test_files/encoding.slt | 144 +-
datafusion/sqllogictest/test_files/errors.slt | 4 +-
datafusion/sqllogictest/test_files/explain.slt | 30 +-
.../sqllogictest/test_files/explain_analyze.slt | 41 +
.../sqllogictest/test_files/explain_tree.slt | 108 +-
datafusion/sqllogictest/test_files/expr.slt | 2 +-
datafusion/sqllogictest/test_files/group_by.slt | 177 +-
datafusion/sqllogictest/test_files/grouping.slt | 12 +
.../sqllogictest/test_files/information_schema.slt | 14 +
datafusion/sqllogictest/test_files/insert.slt | 15 +-
.../sqllogictest/test_files/insert_to_external.slt | 10 +-
datafusion/sqllogictest/test_files/join.slt.part | 9 +-
datafusion/sqllogictest/test_files/joins.slt | 205 +-
datafusion/sqllogictest/test_files/limit.slt | 16 +-
.../test_files/listing_table_statistics.slt | 2 +-
datafusion/sqllogictest/test_files/math.slt | 99 +-
datafusion/sqllogictest/test_files/metadata.slt | 55 +
.../sqllogictest/test_files/named_arguments.slt | 3 +-
datafusion/sqllogictest/test_files/order.slt | 33 +-
datafusion/sqllogictest/test_files/parquet.slt | 4 +
.../sqllogictest/test_files/parquet_statistics.slt | 12 +-
datafusion/sqllogictest/test_files/predicates.slt | 32 +-
.../test_files/preserve_file_partitioning.slt | 596 ++++++
.../sqllogictest/test_files/push_down_filter.slt | 7 +-
datafusion/sqllogictest/test_files/qualify.slt | 25 +-
datafusion/sqllogictest/test_files/repartition.slt | 14 +-
.../sqllogictest/test_files/repartition_scan.slt | 4 +
.../test_files/repartition_subset_satisfaction.slt | 526 +++++
datafusion/sqllogictest/test_files/scalar.slt | 144 +-
.../sqllogictest/test_files/schema_evolution.slt | 144 ++
datafusion/sqllogictest/test_files/select.slt | 13 +-
.../sqllogictest/test_files/set_variable.slt | 32 +
.../sqllogictest/test_files/sort_merge_join.slt | 2 +-
.../sqllogictest/test_files/sort_pushdown.slt | 886 ++++++++
.../test_files/spark/aggregate/try_sum.slt | 140 ++
.../test_files/spark/datetime/hour.slt | 23 +-
.../test_files/spark/datetime/minute.slt | 23 +-
.../test_files/spark/datetime/second.slt | 23 +-
.../sqllogictest/test_files/spark/math/hex.slt | 15 +
datafusion/sqllogictest/test_files/struct.slt | 171 ++
datafusion/sqllogictest/test_files/subquery.slt | 52 +-
.../sqllogictest/test_files/subquery_sort.slt | 9 +-
.../sqllogictest/test_files/table_functions.slt | 24 +
datafusion/sqllogictest/test_files/topk.slt | 2 +-
.../sqllogictest/test_files/tpch/plans/q1.slt.part | 11 +-
.../test_files/tpch/plans/q10.slt.part | 41 +-
.../test_files/tpch/plans/q11.slt.part | 65 +-
.../test_files/tpch/plans/q12.slt.part | 19 +-
.../test_files/tpch/plans/q13.slt.part | 23 +-
.../test_files/tpch/plans/q14.slt.part | 12 +-
.../test_files/tpch/plans/q15.slt.part | 29 +-
.../test_files/tpch/plans/q16.slt.part | 40 +-
.../test_files/tpch/plans/q17.slt.part | 25 +-
.../test_files/tpch/plans/q18.slt.part | 40 +-
.../test_files/tpch/plans/q19.slt.part | 16 +-
.../sqllogictest/test_files/tpch/plans/q2.slt.part | 103 +-
.../test_files/tpch/plans/q20.slt.part | 56 +-
.../test_files/tpch/plans/q21.slt.part | 64 +-
.../test_files/tpch/plans/q22.slt.part | 43 +-
.../sqllogictest/test_files/tpch/plans/q3.slt.part | 35 +-
.../sqllogictest/test_files/tpch/plans/q4.slt.part | 21 +-
.../sqllogictest/test_files/tpch/plans/q5.slt.part | 63 +-
.../sqllogictest/test_files/tpch/plans/q7.slt.part | 69 +-
.../sqllogictest/test_files/tpch/plans/q8.slt.part | 89 +-
.../sqllogictest/test_files/tpch/plans/q9.slt.part | 63 +-
datafusion/sqllogictest/test_files/union.slt | 122 +-
datafusion/sqllogictest/test_files/unnest.slt | 17 +-
datafusion/sqllogictest/test_files/update.slt | 20 +-
datafusion/sqllogictest/test_files/window.slt | 220 +-
.../sqllogictest/test_files/window_limits.slt | 10 +-
datafusion/substrait/Cargo.toml | 2 +-
datafusion/substrait/src/extensions.rs | 6 +-
datafusion/substrait/src/lib.rs | 1 +
.../consumer/expr/aggregate_function.rs | 6 +-
.../src/logical_plan/consumer/expr/cast.rs | 4 +-
.../logical_plan/consumer/expr/field_reference.rs | 4 +-
.../consumer/expr/function_arguments.rs | 4 +-
.../src/logical_plan/consumer/expr/literal.rs | 48 +-
.../src/logical_plan/consumer/expr/mod.rs | 22 +-
.../logical_plan/consumer/expr/scalar_function.rs | 25 +-
.../logical_plan/consumer/expr/singular_or_list.rs | 4 +-
.../src/logical_plan/consumer/expr/subquery.rs | 8 +-
.../logical_plan/consumer/expr/window_function.rs | 10 +-
.../substrait/src/logical_plan/consumer/plan.rs | 67 +-
.../src/logical_plan/consumer/rel/aggregate_rel.rs | 8 +-
.../src/logical_plan/consumer/rel/exchange_rel.rs | 4 +-
.../src/logical_plan/consumer/rel/fetch_rel.rs | 6 +-
.../src/logical_plan/consumer/rel/join_rel.rs | 6 +-
.../substrait/src/logical_plan/consumer/rel/mod.rs | 6 +-
.../src/logical_plan/consumer/rel/project_rel.rs | 4 +-
.../src/logical_plan/consumer/rel/read_rel.rs | 20 +-
.../src/logical_plan/consumer/rel/sort_rel.rs | 2 +-
.../logical_plan/consumer/substrait_consumer.rs | 12 +-
.../substrait/src/logical_plan/consumer/types.rs | 40 +-
.../substrait/src/logical_plan/consumer/utils.rs | 9 +-
.../producer/expr/aggregate_function.rs | 2 +-
.../src/logical_plan/producer/expr/cast.rs | 6 +-
.../logical_plan/producer/expr/field_reference.rs | 6 +-
.../src/logical_plan/producer/expr/if_then.rs | 2 +-
.../src/logical_plan/producer/expr/literal.rs | 8 +-
.../src/logical_plan/producer/expr/mod.rs | 8 +-
.../logical_plan/producer/expr/scalar_function.rs | 14 +-
.../logical_plan/producer/expr/singular_or_list.rs | 2 +-
.../src/logical_plan/producer/expr/subquery.rs | 2 +-
.../logical_plan/producer/expr/window_function.rs | 10 +-
.../substrait/src/logical_plan/producer/plan.rs | 6 +-
.../src/logical_plan/producer/rel/aggregate_rel.rs | 12 +-
.../src/logical_plan/producer/rel/exchange_rel.rs | 6 +-
.../src/logical_plan/producer/rel/fetch_rel.rs | 2 +-
.../src/logical_plan/producer/rel/join.rs | 6 +-
.../src/logical_plan/producer/rel/project_rel.rs | 4 +-
.../src/logical_plan/producer/rel/read_rel.rs | 13 +-
.../src/logical_plan/producer/rel/set_rel.rs | 2 +-
.../src/logical_plan/producer/rel/sort_rel.rs | 4 +-
.../logical_plan/producer/substrait_producer.rs | 14 +-
.../substrait/src/logical_plan/producer/types.rs | 10 +-
.../substrait/src/logical_plan/producer/utils.rs | 4 +-
datafusion/substrait/src/physical_plan/consumer.rs | 26 +-
datafusion/substrait/src/physical_plan/producer.rs | 157 +-
datafusion/substrait/src/serializer.rs | 1 +
.../substrait/tests/cases/aggregation_tests.rs | 16 +-
.../substrait/tests/cases/consumer_integration.rs | 420 ++--
.../substrait/tests/cases/emit_kind_tests.rs | 44 +-
datafusion/substrait/tests/cases/function_test.rs | 8 +-
datafusion/substrait/tests/cases/logical_plans.rs | 65 +-
.../tests/cases/roundtrip_logical_plan.rs | 110 +-
.../tests/cases/roundtrip_physical_plan.rs | 2 +-
datafusion/substrait/tests/cases/serialize.rs | 20 +-
.../substrait/tests/cases/substrait_validations.rs | 24 +-
datafusion/substrait/tests/utils.rs | 22 +-
datafusion/wasmtest/Cargo.toml | 2 +-
datafusion/wasmtest/src/lib.rs | 2 +-
dev/rust_lint.sh | 1 +
docs/requirements.txt | 4 +-
docs/source/_static/theme_overrides.css | 18 +
docs/source/contributor-guide/architecture.md | 2 +-
docs/source/index.rst | 1 +
.../library-user-guide/extending-operators.md | 7 +-
docs/source/library-user-guide/extending-sql.md | 339 +++
.../library-user-guide/functions/adding-udfs.md | 4 +-
docs/source/library-user-guide/query-optimizer.md | 18 +-
docs/source/library-user-guide/upgrading.md | 311 ++-
docs/source/user-guide/cli/functions.md | 32 +
docs/source/user-guide/configs.md | 249 +--
docs/source/user-guide/sql/ddl.md | 2 +-
docs/source/user-guide/sql/format_options.md | 38 +-
docs/source/user-guide/sql/scalar_functions.md | 188 +-
rust-toolchain.toml | 2 +-
rustfmt.toml | 2 +-
test-utils/src/array_gen/binary.rs | 2 +-
test-utils/src/array_gen/boolean.rs | 2 +-
test-utils/src/array_gen/decimal.rs | 2 +-
test-utils/src/array_gen/primitive.rs | 4 +-
test-utils/src/array_gen/random_data.rs | 16 +-
test-utils/src/array_gen/string.rs | 2 +-
test-utils/src/string_gen.rs | 2 +-
1022 files changed, 57026 insertions(+), 23855 deletions(-)
copy benchmarks/queries/clickbench/queries/{ => sorted_data}/q0.sql (75%)
create mode 100644 benchmarks/src/smj.rs
create mode 100755 ci/scripts/check_examples_docs.sh
create mode 100755 ci/scripts/doc_prettier_check.sh
create mode 100644 datafusion-examples/examples/dataframe/cache_factory.rs
create mode 100644 datafusion-examples/examples/relation_planner/main.rs
create mode 100644
datafusion-examples/examples/relation_planner/match_recognize.rs
create mode 100644
datafusion-examples/examples/relation_planner/pivot_unpivot.rs
create mode 100644
datafusion-examples/examples/relation_planner/table_sample.rs
create mode 100644 datafusion-examples/examples/sql_ops/custom_sql_parser.rs
delete mode 100644 datafusion-examples/examples/sql_ops/dialect.rs
create mode 100644 datafusion/common/benches/with_hashes.rs
create mode 100644 datafusion/core/benches/preserve_file_partitioning.rs
create mode 100644 datafusion/core/benches/range_and_generate_series.rs
create mode 100644 datafusion/core/tests/custom_sources_cases/dml_planning.rs
rename datafusion/core/tests/parquet/{schema_adapter.rs => expr_adapter.rs}
(69%)
create mode 100644 datafusion/core/tests/physical_optimizer/pushdown_sort.rs
delete mode 100644 datafusion/core/tests/schema_adapter/mod.rs
delete mode 100644
datafusion/core/tests/schema_adapter/schema_adapter_integration_tests.rs
create mode 100644 datafusion/core/tests/sql/unparser.rs
create mode 100644 datafusion/core/tests/user_defined/relation_planner.rs
create mode 100644 datafusion/datasource-parquet/src/sort.rs
copy datafusion/execution/src/cache/{cache_unit.rs => file_metadata_cache.rs}
(85%)
create mode 100644 datafusion/ffi/src/proto/logical_extension_codec.rs
copy datafusion/{core/tests/catalog_listing => ffi/src/proto}/mod.rs (92%)
create mode 100644 datafusion/ffi/src/proto/physical_extension_codec.rs
rename datafusion/ffi/src/{session_config.rs => session/config.rs} (87%)
create mode 100644 datafusion/ffi/src/session/mod.rs
create mode 100644 datafusion/ffi/tests/utils/mod.rs
create mode 100644 datafusion/functions/benches/contains.rs
copy datafusion/functions/benches/{upper.rs => crypto.rs} (55%)
create mode 100644 datafusion/functions/benches/ends_with.rs
copy datafusion/functions/benches/{uuid.rs => factorial.rs} (52%)
create mode 100644 datafusion/functions/benches/left.rs
create mode 100644 datafusion/functions/benches/levenshtein.rs
create mode 100644 datafusion/functions/benches/regexp_count.rs
copy datafusion/functions/benches/{repeat.rs => replace.rs} (56%)
create mode 100644 datafusion/functions/benches/starts_with.rs
create mode 100644 datafusion/functions/benches/translate.rs
rename datafusion/functions/benches/{ltrim.rs => trim.rs} (56%)
create mode 100644 datafusion/functions/src/core/arrow_metadata.rs
create mode 100644 datafusion/functions/src/datetime/make_time.rs
create mode 100644 datafusion/functions/src/datetime/to_time.rs
create mode 100644 datafusion/functions/src/math/ceil.rs
create mode 100644 datafusion/functions/src/math/decimal.rs
create mode 100644 datafusion/functions/src/math/floor.rs
rename datafusion/{physical-plan =>
physical-expr-common}/src/metrics/baseline.rs (97%)
rename datafusion/{physical-plan =>
physical-expr-common}/src/metrics/builder.rs (99%)
rename datafusion/{physical-plan =>
physical-expr-common}/src/metrics/custom.rs (98%)
create mode 100644 datafusion/physical-expr-common/src/metrics/expression.rs
rename datafusion/{physical-plan => physical-expr-common}/src/metrics/mod.rs
(96%)
rename datafusion/{physical-plan => physical-expr-common}/src/metrics/value.rs
(99%)
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/boolean_lookup_table.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/bytes_like_lookup_table.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/mod.rs
create mode 100644
datafusion/physical-expr/src/expressions/case/literal_lookup_table/primitive_lookup_table.rs
create mode 100644 datafusion/physical-optimizer/src/pushdown_sort.rs
create mode 100644
datafusion/physical-plan/src/joins/hash_join/inlist_builder.rs
copy datafusion/{core/src/bin/print_config_docs.rs =>
physical-plan/src/metrics.rs} (82%)
create mode 100644 datafusion/physical-plan/src/sort_pushdown.rs
create mode 100644 datafusion/spark/src/function/aggregate/try_sum.rs
create mode 100644 datafusion/spark/src/function/datetime/extract.rs
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_date.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_integer.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_time.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_interval_double.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_interval_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_negate_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_time_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_time_time.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_duration.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_timestamp.slt
rename datafusion/sqllogictest/test_files/{ =>
datetime}/current_date_timezone.slt (100%)
rename datafusion/sqllogictest/test_files/{ =>
datetime}/current_time_timezone.slt (100%)
rename datafusion/sqllogictest/test_files/{expr => datetime}/date_part.slt
(100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/dates.slt (68%)
rename datafusion/sqllogictest/test_files/{ => datetime}/interval.slt (100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/interval_mysql.slt
(100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/timestamps.slt (88%)
create mode 100644 datafusion/sqllogictest/test_files/dml_delete.slt
create mode 100644 datafusion/sqllogictest/test_files/dml_update.slt
create mode 100644
datafusion/sqllogictest/test_files/preserve_file_partitioning.slt
create mode 100644
datafusion/sqllogictest/test_files/repartition_subset_satisfaction.slt
create mode 100644 datafusion/sqllogictest/test_files/sort_pushdown.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/aggregate/try_sum.slt
create mode 100644 docs/source/library-user-guide/extending-sql.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
