[PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-04 Thread via GitHub
mertak-synnada opened a new pull request, #14499: URL: https://github.com/apache/datafusion/pull/14499 ## Which issue does this PR close? Closes #14446. ## Rationale for this change ## What changes are included in this PR? ## Are these chang

[PR] Always use `StringViewArray` as output of `substr` [datafusion]

2025-02-04 Thread via GitHub
Kev1n8 opened a new pull request, #14498: URL: https://github.com/apache/datafusion/pull/14498 ## Which issue does this PR close? Closes #12338 ## Rationale for this change Generate `StringViewArray` whatever input type is for efficiency. ## What change

Re: [PR] [WIP] checks for build wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms closed pull request #14493: [WIP] checks for build wasm URL: https://github.com/apache/datafusion/pull/14493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] [WIP] checks for build wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms commented on PR #14493: URL: https://github.com/apache/datafusion/pull/14493#issuecomment-2635946177 > duplicate with #14494 ? this is a test to prove the issue has nothing to do with my previous PR, I'll close it. -- This is an automated message from the Apache Git Servi

Re: [PR] [WIP] checks for build wasm [datafusion]

2025-02-04 Thread via GitHub
xudong963 commented on PR #14493: URL: https://github.com/apache/datafusion/pull/14493#issuecomment-2635936440 duplicate with #14494 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] fix(ci): build error with wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms commented on PR #14494: URL: https://github.com/apache/datafusion/pull/14494#issuecomment-2635927296 cc @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] Fix link to volcano parallelism paper [datafusion]

2025-02-04 Thread via GitHub
lewiszlw opened a new pull request, #14497: URL: https://github.com/apache/datafusion/pull/14497 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?

[PR] fix: rewrite fetch, skip of the Limit node in correct order [datafusion]

2025-02-04 Thread via GitHub
evenyag opened a new pull request, #14496: URL: https://github.com/apache/datafusion/pull/14496 ## Which issue does this PR close? Closes #. ## Rationale for this change We found a bug related to `with_new_exprs()` for the `Limit` plan in https://github.c

Re: [PR] refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [datafusion]

2025-02-04 Thread via GitHub
Chen-Yuan-Lai closed pull request #14495: refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead URL: https://github.com/apache/datafusion/pull/14495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Browser-accessible official DataFusion playground / DataFusion fiddle [datafusion]

2025-02-04 Thread via GitHub
gabotechs commented on issue #13818: URL: https://github.com/apache/datafusion/issues/13818#issuecomment-2635854786 Made it a while ago and still use it for quickly trying out stuff, there are lots of low hanging fruits for improving it (local storage based query history, syntax/error highl

Re: [I] Project Ideas for GSoC 2025 [datafusion]

2025-02-04 Thread via GitHub
ozankabak commented on issue #14478: URL: https://github.com/apache/datafusion/issues/14478#issuecomment-2635853610 @comphead, would you be willing to mentor a student on a project to study our codebase and dependencies to reduce DF binary size? -- This is an automated message from the Ap

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-04 Thread via GitHub
findepi commented on PR #14268: URL: https://github.com/apache/datafusion/pull/14268#issuecomment-2635850123 > I fully recognize this creates behavior that diverges from PostgreSQL/DuckDB semantics for the various UDFs in this PR. However, there’s a critical distinction: **System contracts

[PR] refactor: replace uses of `arrow_buffer` & `arrow_array` with reexport in arrow [datafusion]

2025-02-04 Thread via GitHub
Chen-Yuan-Lai opened a new pull request, #14495: URL: https://github.com/apache/datafusion/pull/14495 ## Which issue does this PR close? Closes #14115. ## Rationale for this change As [14115 issue comment](https://github.com/apache/datafusion/issues/14115#is

Re: [PR] Require space after -- to start single line comment in MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
iffyio commented on code in PR #1705: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1705#discussion_r1942311180 ## src/dialect/mod.rs: ## @@ -880,6 +880,15 @@ pub trait Dialect: Debug + Any { fn supports_table_hints(&self) -> bool { false } + +

Re: [PR] [WIP] checks for build wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms closed pull request #14493: [WIP] checks for build wasm URL: https://github.com/apache/datafusion/pull/14493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[PR] fix(ci): build error with wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms opened a new pull request, #14494: URL: https://github.com/apache/datafusion/pull/14494 ## Which issue does this PR close? like this https://github.com/apache/datafusion/actions/runs/13151403418/job/36699387339?pr=14493 Closes #. ## Rationale for this chang

[PR] [WIP] checks for build wasm [datafusion]

2025-02-04 Thread via GitHub
Lordworms opened a new pull request, #14493: URL: https://github.com/apache/datafusion/pull/14493 ## Which issue does this PR close? uuid crate wasm-build ci job failed on my pr check if it is related to changes Closes #. ## Rationale for this change ## Wh

Re: [I] Attach `Diagnostic` to "duplicate table name" error [datafusion]

2025-02-04 Thread via GitHub
zjregee commented on issue #14436: URL: https://github.com/apache/datafusion/issues/14436#issuecomment-2635780992 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-04 Thread via GitHub
wForget commented on code in PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329#discussion_r1942247623 ## native/core/src/execution/metrics/utils.rs: ## @@ -55,60 +64,21 @@ pub fn update_comet_metric( Some(metrics.aggregate_by_name()) }; -upd

Re: [PR] Improve performance 10%-100% in `FIRST_VALUE` / `LAST_VALUE` by not sort rows in `FirstValueAccumulator` [datafusion]

2025-02-04 Thread via GitHub
ozankabak commented on PR #14402: URL: https://github.com/apache/datafusion/pull/14402#issuecomment-2635656916 Surgical and effective -- thanks everybdoy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-04 Thread via GitHub
wForget commented on code in PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329#discussion_r1942138342 ## native/core/src/execution/jni_api.rs: ## @@ -508,9 +505,6 @@ pub unsafe extern "system" fn Java_org_apache_comet_Native_executePlan( let next_

Re: [PR] WIP: fix regression after replacing `Vec` with `HashSet` [datafusion]

2025-02-04 Thread via GitHub
github-actions[bot] commented on PR #13656: URL: https://github.com/apache/datafusion/pull/13656#issuecomment-2635531668 Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-04 Thread via GitHub
wForget commented on PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329#issuecomment-2635523066 @andygrove @mbutrovich @parthchandra Thank you for your review and sorry for the late reply. I have just finished my Chinese New Year holiday and will continue this work later.

Re: [PR] Support WITHIN GROUP syntax to standardize certain existing aggregate functions [datafusion]

2025-02-04 Thread via GitHub
Garamda commented on PR #13511: URL: https://github.com/apache/datafusion/pull/13511#issuecomment-2635495223 Thank you @berkaysynnada for your review, and for letting me know about my mistake (submodule changes). I have just reverted submodule changes. I will check if I can add more n

Re: [I] Support "Tracing" / Spans [datafusion]

2025-02-04 Thread via GitHub
erratic-pattern commented on issue #9415: URL: https://github.com/apache/datafusion/issues/9415#issuecomment-2635477969 > I am not clear what additional benefit more direct tracing integration in datafusion would provide, but I may be missing something The `tracing` API is more granul

Re: [PR] disable coercison for unmatched struct type [datafusion]

2025-02-04 Thread via GitHub
Lordworms commented on PR #14409: URL: https://github.com/apache/datafusion/pull/14409#issuecomment-2635465735 @alamb Hi alamb, I have moved the logic in struct_coercion and also fix one bug in coalesce -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] Support "Tracing" / Spans [datafusion]

2025-02-04 Thread via GitHub
erratic-pattern commented on issue #9415: URL: https://github.com/apache/datafusion/issues/9415#issuecomment-2635459901 > I'd like to propose a very simple change first, before going full tracing everywhere: wrapping all task spawn points with .in_current_span(). I agree with this cha

Re: [I] Attach `Diagnostic` to "incompatible type in unary expression" error [datafusion]

2025-02-04 Thread via GitHub
alan910127 commented on issue #14433: URL: https://github.com/apache/datafusion/issues/14433#issuecomment-2635450667 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] feat: add experimental remote HDFS support for native DataFusion reader [datafusion-comet]

2025-02-04 Thread via GitHub
comphead commented on code in PR #1359: URL: https://github.com/apache/datafusion-comet/pull/1359#discussion_r1942084981 ## native/core/Cargo.toml: ## @@ -77,6 +77,7 @@ datafusion-comet-proto = { workspace = true } object_store = { workspace = true } url = { workspace = true }

[I] Add separate HDFS submodule to Comet [datafusion-comet]

2025-02-04 Thread via GitHub
comphead opened a new issue, #1368: URL: https://github.com/apache/datafusion-comet/issues/1368 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] fix: `List` of `FixedSizeList` coercion issue in SQL [datafusion]

2025-02-04 Thread via GitHub
alan910127 commented on PR #14468: URL: https://github.com/apache/datafusion/pull/14468#issuecomment-2635404952 > Thank you for this contribution @alan910127 ❤️ Thank you, @alamb! 😊 Glad to contribute! Let me know if there's anything else I can improve. -- This is an automated me

Re: [PR] Improve performance 10%-100% in `FIRST_VALUE` / `LAST_VALUE` by not sort rows in `FirstValueAccumulator` [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 commented on PR #14402: URL: https://github.com/apache/datafusion/pull/14402#issuecomment-2635377541 Thanks @blaginin @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] `FirstValue` / `LastValue` performance optimisations [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 closed issue #14215: `FirstValue` / `LastValue` performance optimisations URL: https://github.com/apache/datafusion/issues/14215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Improve performance 10%-100% in `FIRST_VALUE` / `LAST_VALUE` by not sort rows in `FirstValueAccumulator` [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 merged PR #14402: URL: https://github.com/apache/datafusion/pull/14402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942056824 ## datafusion/expr-common/src/signature.rs: ## @@ -213,9 +221,8 @@ pub enum TypeSignatureClass { Interval, Duration, Native(LogicalTypeRef), -

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942056824 ## datafusion/expr-common/src/signature.rs: ## @@ -213,9 +221,8 @@ pub enum TypeSignatureClass { Interval, Duration, Native(LogicalTypeRef), -

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-04 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942056824 ## datafusion/expr-common/src/signature.rs: ## @@ -213,9 +221,8 @@ pub enum TypeSignatureClass { Interval, Duration, Native(LogicalTypeRef), -

[PR] Require space after -- to start single line comment in MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
hansott opened a new pull request, #1705: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1705 - Docs https://dev.mysql.com/doc/refman/8.4/en/ansi-diff-comments.html - Lexer https://github.com/mysql/mysql-server/blob/trunk/sql/sql_lex.cc#L1465-L1472 - Char checks ```cpp

Re: [PR] fix: Limits are not applied correctly [datafusion]

2025-02-04 Thread via GitHub
adriangb commented on PR #14418: URL: https://github.com/apache/datafusion/pull/14418#issuecomment-2635221811 Thank you all for the quick fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat: add experimental remote HDFS support for native DataFusion reader [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura commented on code in PR #1359: URL: https://github.com/apache/datafusion-comet/pull/1359#discussion_r1941967578 ## Makefile: ## @@ -95,7 +98,7 @@ release-linux: clean cd native && RUSTFLAGS="-Ctarget-cpu=native -Ctarget-feature=-prefer-256-bit" cargo bu

Re: [PR] Improve performance 10%-100% in `FIRST_VALUE` / `LAST_VALUE` by not sort rows in `FirstValueAccumulator` [datafusion]

2025-02-04 Thread via GitHub
alamb commented on code in PR #14402: URL: https://github.com/apache/datafusion/pull/14402#discussion_r1941935727 ## datafusion/functions-aggregate/src/first_last.rs: ## @@ -259,19 +260,17 @@ impl FirstValueAccumulator { }) .collect::>(); -if

Re: [PR] Require space after -- to start single line comment in MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
hansott closed pull request #1704: Require space after -- to start single line comment in MySQL URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1704 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[PR] Require space after -- to start single line comment in MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
hansott opened a new pull request, #1704: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1704 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Collection of errors [datafusion]

2025-02-04 Thread via GitHub
eliaperantoni commented on PR #14439: URL: https://github.com/apache/datafusion/pull/14439#issuecomment-2635144203 Thanks @comphead for your feedback. We appreciate it 🙏 > What comes to my mind is the PR might make the output too cumbersome The actual error output doesn't change

[PR] fix(doc): remove AWS_PROFILE from supported S3 configuration [datafusion]

2025-02-04 Thread via GitHub
hussein-awala opened a new pull request, #14492: URL: https://github.com/apache/datafusion/pull/14492 This PR updates `datasources.md` doc by removing `AWS_PROFILE` from the supported S3 configuration as its experimental support was removed from `object_store` crate in the version `0.6.0`:

Re: [PR] Add support for DISTINCT + ORDER BY in ARRAY_AGG [datafusion]

2025-02-04 Thread via GitHub
alamb closed pull request #14413: Add support for DISTINCT + ORDER BY in ARRAY_AGG URL: https://github.com/apache/datafusion/pull/14413 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Browser-accessible official DataFusion playground / DataFusion fiddle [datafusion]

2025-02-04 Thread via GitHub
alamb commented on issue #13818: URL: https://github.com/apache/datafusion/issues/13818#issuecomment-2635134931 I just game across one from @gabotechs : https://datafusion-fiddle.vercel.app/ Repo: https://github.com/gabotechs/datafusion-fiddle -- This is an automated message from t

Re: [PR] Add support for DISTINCT + ORDER BY in ARRAY_AGG [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14413: URL: https://github.com/apache/datafusion/pull/14413#issuecomment-2635132402 Close/reopen to rerun CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] STRING_AGG missing functionality [datafusion]

2025-02-04 Thread via GitHub
alamb closed pull request #14412: STRING_AGG missing functionality URL: https://github.com/apache/datafusion/pull/14412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Comet possibly preventing AQE optimization [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura commented on issue #1266: URL: https://github.com/apache/datafusion-comet/issues/1266#issuecomment-2635123837 Perhaps this SPARK-50258 is for Spark 4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Collection of errors [datafusion]

2025-02-04 Thread via GitHub
mkarbo commented on PR #14439: URL: https://github.com/apache/datafusion/pull/14439#issuecomment-2635090710 > Thanks @eliaperantoni > > What comes to my mind is the PR might make the output too cumbersome, and some time back I think we already faced the issue that extra attention to

Re: [PR] disable coercison for unmatched struct type [datafusion]

2025-02-04 Thread via GitHub
Lordworms commented on PR #14409: URL: https://github.com/apache/datafusion/pull/14409#issuecomment-2635087583 > struct_coercion Sure, I can do that > Hi @Lordworms -- thank you very much for this PR and for working on the issue > > While reviewing this PR I noticed a fe

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-04 Thread via GitHub
alamb commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1941893809 ## README.md: ## @@ -152,26 +152,13 @@ deprecate methods before removing them, according to the [deprecation guidelines [deprecation guidelines]: https://datafu

Re: [PR] disable coercison for unmatched struct type [datafusion]

2025-02-04 Thread via GitHub
alamb commented on code in PR #14409: URL: https://github.com/apache/datafusion/pull/14409#discussion_r1941881603 ## datafusion/optimizer/src/analyzer/type_coercion.rs: ## @@ -873,6 +872,21 @@ fn coerce_case_expression(case: Case, schema: &DFSchema) -> Result { })

Re: [PR] fix: pass scale to DF round in spark_round [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura commented on code in PR #1341: URL: https://github.com/apache/datafusion-comet/pull/1341#discussion_r1941886037 ## native/spark-expr/src/math_funcs/round.rs: ## @@ -135,3 +136,50 @@ fn decimal_round_f(scale: &i8, point: &i64) -> Box i128> { Box::new(mov

Re: [I] Scale argument is not passed to DF `round` in `spark_round` [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura closed issue #1340: Scale argument is not passed to DF `round` in `spark_round` URL: https://github.com/apache/datafusion-comet/issues/1340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] fix: pass scale to DF round in spark_round [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura merged PR #1341: URL: https://github.com/apache/datafusion-comet/pull/1341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsub

Re: [I] Scale argument is not passed to DF `round` in `spark_round` [datafusion-comet]

2025-02-04 Thread via GitHub
kazuyukitanimura closed issue #1340: Scale argument is not passed to DF `round` in `spark_round` URL: https://github.com/apache/datafusion-comet/issues/1340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Define extension API for user-defined invariants. [datafusion]

2025-02-04 Thread via GitHub
alamb commented on issue #14029: URL: https://github.com/apache/datafusion/issues/14029#issuecomment-2635031533 Thanks again @wiedld -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] DFParser should skip unsupported COPY INTO [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14382: URL: https://github.com/apache/datafusion/pull/14382#issuecomment-2635032219 Thanks again @osipovartem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] DFParser cannot parse CopyIntoSnowflake [datafusion]

2025-02-04 Thread via GitHub
alamb closed issue #14372: DFParser cannot parse CopyIntoSnowflake URL: https://github.com/apache/datafusion/issues/14372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Improve Unparser (scalar_to_sql) to respect dialect timestamp type overrides [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14407: URL: https://github.com/apache/datafusion/pull/14407#issuecomment-2635032807 Thanks again @sgrebnov and @phillipleblanc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-04 Thread via GitHub
shehabgamin commented on PR #14268: URL: https://github.com/apache/datafusion/pull/14268#issuecomment-2635031823 > @jayzhan211 To keep it simple ill just remove AnyNative and use coerce_types so we don't block this PR any longer. We can have a larger discussion and align on goals afterwards

Re: [PR] Improve Unparser (scalar_to_sql) to respect dialect timestamp type overrides [datafusion]

2025-02-04 Thread via GitHub
alamb merged PR #14407: URL: https://github.com/apache/datafusion/pull/14407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] DFParser should skip unsupported COPY INTO [datafusion]

2025-02-04 Thread via GitHub
alamb merged PR #14382: URL: https://github.com/apache/datafusion/pull/14382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Provide user-defined invariants for logical node extensions. [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14329: URL: https://github.com/apache/datafusion/pull/14329#issuecomment-2635029104 Thanks again @wiedld -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] Define extension API for user-defined invariants. [datafusion]

2025-02-04 Thread via GitHub
alamb closed issue #14029: Define extension API for user-defined invariants. URL: https://github.com/apache/datafusion/issues/14029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] Define extension API for user-defined invariants. [datafusion]

2025-02-04 Thread via GitHub
alamb commented on issue #14029: URL: https://github.com/apache/datafusion/issues/14029#issuecomment-2635031142 I think with https://github.com/apache/datafusion/pull/14329 we have completed the planned work for this ticket. Let's file another one to track any additional work we may

Re: [PR] Provide user-defined invariants for logical node extensions. [datafusion]

2025-02-04 Thread via GitHub
alamb merged PR #14329: URL: https://github.com/apache/datafusion/pull/14329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [I] Implement xxhash algorithms as part of the expression API [datafusion]

2025-02-04 Thread via GitHub
Spaarsh commented on issue #14044: URL: https://github.com/apache/datafusion/issues/14044#issuecomment-2634981559 @HectorPascual I have added an optional seed argument and have also added support for Binary input. -- This is an automated message from the Apache Git Service. To respond to

Re: [I] Write "upgrade guide" for DataFusion 44.0.0 [datafusion]

2025-02-04 Thread via GitHub
andygrove closed issue #13702: Write "upgrade guide" for DataFusion 44.0.0 URL: https://github.com/apache/datafusion/issues/13702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Write "upgrade guide" for DataFusion 44.0.0 [datafusion]

2025-02-04 Thread via GitHub
andygrove commented on issue #13702: URL: https://github.com/apache/datafusion/issues/13702#issuecomment-2634978534 Yes, I'll go ahead and close this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Parse SET GLOBAL variable modifier for MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
mvzink closed pull request #1696: Parse SET GLOBAL variable modifier for MySQL URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Write "upgrade guide" for DataFusion 44.0.0 [datafusion]

2025-02-04 Thread via GitHub
Omega359 commented on issue #13702: URL: https://github.com/apache/datafusion/issues/13702#issuecomment-2634958553 @andygrove - can this issue be closed now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Parse MySQL `SET GLOBAL` variables [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
mvzink commented on issue #1694: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1694#issuecomment-2634948923 As mentioned in #1696, this may be best accomplished in combination with #1697; possibly this issue should just be closed in favor of that one. -- This is an automa

Re: [PR] Parse SET GLOBAL variable modifier for MySQL [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
mvzink commented on PR #1696: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1696#issuecomment-2634944733 I actually think between the point about API scalability and the further/future need for substantially different parsing for MySQL (#1697), this particular approach that j

Re: [PR] 14044/enhancement/add xxhash algorithms in expression api [datafusion]

2025-02-04 Thread via GitHub
Spaarsh commented on PR #14367: URL: https://github.com/apache/datafusion/pull/14367#issuecomment-2634910919 I will add support for Null values and write the tests as well! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] fix: `List` of `FixedSizeList` coercion issue in SQL [datafusion]

2025-02-04 Thread via GitHub
alamb commented on code in PR #14468: URL: https://github.com/apache/datafusion/pull/14468#discussion_r1941729353 ## datafusion/sqllogictest/test_files/array.slt: ## @@ -6862,9 +6862,8 @@ select make_array(f0) from fixed_size_list_array query T select arrow_typeof(make_array(f

Re: [I] Jan 18, 2025: This week(s) in DataFusion [datafusion]

2025-02-04 Thread via GitHub
alamb commented on issue #14179: URL: https://github.com/apache/datafusion/issues/14179#issuecomment-2634895592 Next week (s): - https://github.com/apache/datafusion/issues/14491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [I] Jan 18, 2025: This week(s) in DataFusion [datafusion]

2025-02-04 Thread via GitHub
alamb closed issue #14179: Jan 18, 2025: This week(s) in DataFusion URL: https://github.com/apache/datafusion/issues/14179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[I] Feb 4, 2025: This week(s) in DataFusion [datafusion]

2025-02-04 Thread via GitHub
alamb opened a new issue, #14491: URL: https://github.com/apache/datafusion/issues/14491 ### Is your feature request related to a problem or challenge? ## Introduction This ticket is my weekly-ish summary of interesting things happening in DataFusion. Note this is not a complete li

[PR] Parse Postgres VARBIT datatype [datafusion-sqlparser-rs]

2025-02-04 Thread via GitHub
mvzink opened a new pull request, #1703: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1703 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Serialize `parquet_options` in `datafusion-proto` [datafusion]

2025-02-04 Thread via GitHub
robtandy commented on PR #14465: URL: https://github.com/apache/datafusion/pull/14465#issuecomment-2634888124 Amazing! Thank you so much @blaginin ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] datafusion-ray is not published to pypi.org [datafusion-ray]

2025-02-04 Thread via GitHub
robtandy commented on issue #59: URL: https://github.com/apache/datafusion-ray/issues/59#issuecomment-2634885083 👋 Hi @sairamkrish ! I've been working on a revised version of datafusion ray for the past month or so. I submitted a PR today in fact. It replaces the disk based shuf

Re: [PR] chore: clean up dependencies for datafusion cli [datafusion]

2025-02-04 Thread via GitHub
comphead merged PR #14484: URL: https://github.com/apache/datafusion/pull/14484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [I] Use `NullBufferBuilder` instead of `BooleanBufferBuilder` for creating Null masks [datafusion]

2025-02-04 Thread via GitHub
alamb commented on issue #14115: URL: https://github.com/apache/datafusion/issues/14115#issuecomment-2634836571 BTW @Chen-Yuan-Lai -- https://github.com/apache/arrow-rs/releases/54.1.0 -- This is an automated message from the Apache Git Service. To respond to the message, please l

[I] Use `NullBufferBuilder` instead of `BooleanBufferBuilder` for creating Null masks [datafusion]

2025-02-04 Thread via GitHub
alamb opened a new issue, #14115: URL: https://github.com/apache/datafusion/issues/14115 ### Is your feature request related to a problem or challenge? DataFusion uses `BooleanBuffer` in several places to create Null buffers. I thought there was a clever optimization for handling data

Re: [I] to_unixtime does not support timestamps with a timezone [datafusion]

2025-02-04 Thread via GitHub
Omega359 commented on issue #14488: URL: https://github.com/apache/datafusion/issues/14488#issuecomment-2634815551 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Minor: `cargo fmt` to fix CI [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14487: URL: https://github.com/apache/datafusion/pull/14487#issuecomment-2634830107 Sorry again for the over eager clicking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Minor: `cargo fmt` to fix CI [datafusion]

2025-02-04 Thread via GitHub
alamb merged PR #14487: URL: https://github.com/apache/datafusion/pull/14487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-04 Thread via GitHub
rkrishn7 commented on PR #14472: URL: https://github.com/apache/datafusion/pull/14472#issuecomment-2634824171 Thanks @alamb. Will fix it up when I'm back from work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Collection of errors [datafusion]

2025-02-04 Thread via GitHub
comphead commented on PR #14439: URL: https://github.com/apache/datafusion/pull/14439#issuecomment-2634819291 Thanks @eliaperantoni What comes to my mind is the PR might make the output too cumbersome, and some time back I think we already faced the issue that extra attention to error c

[PR] to_unixtime does not support timestamps with a timezone [datafusion]

2025-02-04 Thread via GitHub
Omega359 opened a new pull request, #14490: URL: https://github.com/apache/datafusion/pull/14490 ## Which issue does this PR close? Closes #14488 ## Rationale for this change Fix functionality gap. ## What changes are included in this PR? Code, slt t

Re: [PR] bug: Fix edge cases in array_slice [datafusion]

2025-02-04 Thread via GitHub
jkosh44 commented on code in PR #14489: URL: https://github.com/apache/datafusion/pull/14489#discussion_r1941739956 ## datafusion/functions-nested/src/extract.rs: ## @@ -487,7 +487,17 @@ where // 0 ~ len - 1 let adjusted_zero_index = if index < 0 {

Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14472: URL: https://github.com/apache/datafusion/pull/14472#issuecomment-2634788395 Thank you @rkrishn7 -- I marked this PR as a draft as it seems a CI test is failing. Thank you! -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14472: URL: https://github.com/apache/datafusion/pull/14472#issuecomment-2634788870 FYI @gatesn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[PR] bug: Fix edge cases in array_slice [datafusion]

2025-02-04 Thread via GitHub
jkosh44 opened a new pull request, #14489: URL: https://github.com/apache/datafusion/pull/14489 This commit fixes the following edge cases in the array_slice function so that it's semantics match DuckDB: - When begin < 0 and -begin > length, begin is clamped to the beginning of the

Re: [PR] adding `RowsReader` and writer [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14149: URL: https://github.com/apache/datafusion/pull/14149#issuecomment-2634801071 Converting to a draft to signal there is still some planned work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[I] to_unixtime does not support timestamps with a timezone [datafusion]

2025-02-04 Thread via GitHub
Omega359 opened a new issue, #14488: URL: https://github.com/apache/datafusion/issues/14488 ### Describe the bug Somewhat related to #12892 but different. The following query fails in DF 45: ```sql select to_unixtime(arrow_cast(to_timestamp('2023-01-14T01:01:30'), 'Timestamp

Re: [PR] Run test in all examples documentation [datafusion]

2025-02-04 Thread via GitHub
alamb commented on PR #14485: URL: https://github.com/apache/datafusion/pull/14485#issuecomment-2634796985 😍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

  1   2   3   >