2010YOUY01 commented on code in PR #13540:
URL: https://github.com/apache/datafusion/pull/13540#discussion_r1864147572
##
datafusion/functions-table/src/generate_series.rs:
##
@@ -0,0 +1,180 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributo
demetribu commented on code in PR #1552:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1552#discussion_r1864152593
##
src/dialect/mod.rs:
##
@@ -687,6 +687,41 @@ pub trait Dialect: Debug + Any {
fn is_reserved_for_identifier(&self, kw: Keyword) -> bool {
2010YOUY01 commented on PR #13540:
URL: https://github.com/apache/datafusion/pull/13540#issuecomment-2508873495
> Thank you @2010YOUY01. I reviewed the changes and LGTM. I have a few minor
comments and one question: I noticed another approach of `generate_series()`,
which can be used like t
2010YOUY01 commented on code in PR #13540:
URL: https://github.com/apache/datafusion/pull/13540#discussion_r1864147810
##
datafusion/physical-plan/src/memory.rs:
##
@@ -365,8 +366,165 @@ impl RecordBatchStream for MemoryStream {
}
}
+pub trait StreamingBatchGenerator: Se
2010YOUY01 commented on code in PR #13540:
URL: https://github.com/apache/datafusion/pull/13540#discussion_r1864147468
##
datafusion/core/src/execution/session_state_defaults.rs:
##
@@ -119,6 +120,11 @@ impl SessionStateDefaults {
functions_window::all_default_window_fu
akurmustafa opened a new pull request, #13608:
URL: https://github.com/apache/datafusion/pull/13608
## Which issue does this PR close?
Closes #.
## Rationale for this change
With rust release `1.82.0` `is_sorted` util is available for iterators. This
PR replaces exis
iffyio commented on code in PR #1552:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1552#discussion_r1864122905
##
src/dialect/mod.rs:
##
@@ -687,6 +687,41 @@ pub trait Dialect: Debug + Any {
fn is_reserved_for_identifier(&self, kw: Keyword) -> bool {
andygrove merged PR #1143:
URL: https://github.com/apache/datafusion-ballista/pull/1143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
github-actions[bot] commented on PR #12631:
URL: https://github.com/apache/datafusion/pull/12631#issuecomment-2508785091
Thank you for your contribution. Unfortunately, this pull request is stale
because it has been open 60 days with no activity. Please remove the stale
label or comment or
Weijun-H commented on code in PR #13605:
URL: https://github.com/apache/datafusion/pull/13605#discussion_r1864040114
##
datafusion/sqllogictest/test_files/errors.slt:
##
@@ -70,7 +70,7 @@ SELECT COUNT(*) FROM nonexistentschema.aggregate_test_100
statement error Error during pla
edmondop opened a new pull request, #50:
URL: https://github.com/apache/datafusion-ray/pull/50
Looking at #42 I think we should fix this before modifying the Python code.
I was surprised to see the second test succeeding and the first failing btw
--
This is an automated message from the A
jayzhan211 commented on PR #13592:
URL: https://github.com/apache/datafusion/pull/13592#issuecomment-2508760565
Thanks @findepi @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
jayzhan211 merged PR #13592:
URL: https://github.com/apache/datafusion/pull/13592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dat
akurmustafa merged PR #13554:
URL: https://github.com/apache/datafusion/pull/13554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@da
viirya merged PR #1124:
URL: https://github.com/apache/datafusion-comet/pull/1124
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dat
viirya commented on PR #1124:
URL:
https://github.com/apache/datafusion-comet/pull/1124#issuecomment-2508703710
Thanks @andygrove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
milenkovicm commented on issue #1142:
URL:
https://github.com/apache/datafusion-ballista/issues/1142#issuecomment-2508681263
Draft patch to illustrate "Possible Solution (I)", for `datafusion-python`
(v42) which would solve (py)ballista issues:
```diff
diff --git a/Cargo.lock b/Ca
LeslieKid commented on PR #13607:
URL: https://github.com/apache/datafusion/pull/13607#issuecomment-2508640484
Major changes currently:
- Compare arrays with indices (in a for loop) without `take+eq`.
- Update a single boolean buffer instead of create a new one every time.
I am t
philipcristiano opened a new pull request, #1573:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1573
Extract CreateFunction from Statement enum into it's own struct.
Continue moving structs as part of
https://github.com/apache/datafusion-sqlparser-rs/issues/1204
--
Th
LeslieKid commented on issue #13275:
URL: https://github.com/apache/datafusion/issues/13275#issuecomment-2508540983
> 🤔 I think maybe we can indeed try some solutions without `take` in join?
@LeslieKid seems trying it.
Yes, I am working on comparing arrays with indices in a for-loop (
LeslieKid opened a new pull request, #13607:
URL: https://github.com/apache/datafusion/pull/13607
## Which issue does this PR close?
Closes #12131 .
## Rationale for this change
## What changes are included in this PR?
## Are these changes t
tustvold commented on issue #13525:
URL: https://github.com/apache/datafusion/issues/13525#issuecomment-2508217233
R.e. WASM32
* Arrow support for 32-bit architectures -
https://github.com/apache/arrow-rs/issues/6681
* Object Store support for 32-bit architectures -
https://github
findepi commented on PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#issuecomment-2508177283
> * The IEEE-754 basic arithmetic operations are reproducible.
> ...
> * Floating-point arithmetic is not associative.
These two points imply that a database's Floating-po
findepi commented on code in PR #13592:
URL: https://github.com/apache/datafusion/pull/13592#discussion_r1863791357
##
datafusion/expr/src/expr_fn.rs:
##
@@ -434,10 +434,22 @@ impl SimpleScalarUDF {
volatility: Volatility,
fun: ScalarFunctionImplementation,
andygrove opened a new pull request, #1128:
URL: https://github.com/apache/datafusion-comet/pull/1128
## Which issue does this PR close?
N/A
## Rationale for this change
This is a subset of https://github.com/apache/datafusion-comet/pull/,
separated o
andygrove merged PR #955:
URL: https://github.com/apache/datafusion-python/pull/955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@d
Omega359 commented on issue #13470:
URL: https://github.com/apache/datafusion/issues/13470#issuecomment-2508137692
I came across [this paper](https://dl.acm.org/doi/10.1145/3654991) which I
found interesting for join testing. It won't work for DF yet since DF doesn't
support query hints (at
milenkovicm opened a new pull request, #1143:
URL: https://github.com/apache/datafusion-ballista/pull/1143
Fix clippy issues after rust update (1.83.0),
+ change a logger from warn to debug, as we cant do much with those errors
at the moment and they happen even with core datafusion c
etolbakov commented on PR #1011:
URL:
https://github.com/apache/datafusion-ballista/pull/1011#issuecomment-2508119888
@milenkovicm makes sense! Thanks!
I’m already in the discord, will check the ballista stream.
Happy if you close the issue.
--
This is an automated message from the
milenkovicm commented on PR #1011:
URL:
https://github.com/apache/datafusion-ballista/pull/1011#issuecomment-2508116445
As project was not actively maintained and lot of work accumulated we decide
to change project's scope and remove code so maintainers have smaller code base
to maintain (
etolbakov commented on PR #1011:
URL:
https://github.com/apache/datafusion-ballista/pull/1011#issuecomment-2508078694
@milenkovicm @Dandandan
sorry for the confusion, as per description I was exploring the code and
came across the TODO about the csv file format support. So decided to ad
leoyvens commented on PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#issuecomment-2508063268
There are myths and truths to floating-point reproducibility across
platforms. Some facts I've gathered while working on this:
1. f32 and f64 in Rust follow IEEE-754.
2. Th
jonahgao commented on code in PR #13605:
URL: https://github.com/apache/datafusion/pull/13605#discussion_r1863695929
##
datafusion/sql/src/planner.rs:
##
@@ -622,24 +622,41 @@ pub fn object_name_to_table_reference(
idents_to_table_reference(idents, enable_normalization)
}
Groennbeck commented on PR #1122:
URL:
https://github.com/apache/datafusion-comet/pull/1122#issuecomment-2508022496
https://github.com/apache/datafusion/pull/13600
Have to wait for this to get into the next version
--
This is an automated message from the Apache Git Service.
To res
jonahgao commented on PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#issuecomment-2508009725
> If we value portability, I'd propose that we switch to `libm`, which is
what I've implemented in the second commit.
I think portability is not necessary, and
[PostgreSQL](
peter-toth commented on PR #13589:
URL: https://github.com/apache/datafusion/pull/13589#issuecomment-2508004060
> I have one minor suggestion: if it doesn’t require much effort, could we
also add a test to ensure that jump does not continue to visit subqueries also?
Even though this behavio
jonahgao commented on code in PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#discussion_r1863646359
##
datafusion/sqllogictest/test_files/aggregate_skip_partial.slt:
##
@@ -261,11 +261,11 @@ SELECT c2, min(c5), max(c5), min(c11), max(c11) FROM
aggregate_test_10
berkaysynnada merged PR #13604:
URL: https://github.com/apache/datafusion/pull/13604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@
jatin510 commented on issue #13601:
URL: https://github.com/apache/datafusion/issues/13601#issuecomment-2507913083
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To u
SalcaE opened a new issue, #1127:
URL: https://github.com/apache/datafusion-comet/issues/1127
### Describe the bug
I'm running Comet 0.4.0 and I can't see the log level info to check if Comet
is enabled for Spark SQL query, which I was able to do with version 0.3.0.
### Steps t
SalcaE opened a new issue, #1126:
URL: https://github.com/apache/datafusion-comet/issues/1126
### Describe the bug
I noticed that the file "comet-spark-spark3.4_2.12-0.3.0.jar" is not present
in the "jars" folder inside the
[image](https://github.com/apache/datafusion-comet/pkgs/cont
alamb commented on code in PR #13592:
URL: https://github.com/apache/datafusion/pull/13592#discussion_r1863528647
##
datafusion/expr/src/expr_fn.rs:
##
@@ -434,10 +434,22 @@ impl SimpleScalarUDF {
volatility: Volatility,
fun: ScalarFunctionImplementation,
alamb closed issue #13419: [substrait] support try_cast
URL: https://github.com/apache/datafusion/issues/13419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e
alamb merged PR #13562:
URL: https://github.com/apache/datafusion/pull/13562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
findepi commented on PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#issuecomment-2507810984
> So I went looking to see if there was a performant but portable float math
library we could use. [libm](https://github.com/rust-lang/libm) seems to be it,
it's what rustc uses when
findepi commented on code in PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#discussion_r1863517475
##
datafusion/sqllogictest/test_files/aggregate_skip_partial.slt:
##
@@ -261,11 +261,11 @@ SELECT c2, min(c5), max(c5), min(c11), max(c11) FROM
aggregate_test_100
alamb opened a new pull request, #13606:
URL: https://github.com/apache/datafusion/pull/13606
## Which issue does this PR close?
- Part of https://github.com/apache/datafusion/issues/13359
## Rationale for this change
While testing https://github.com/apache/datafusion
leoyvens commented on PR #13590:
URL: https://github.com/apache/datafusion/pull/13590#issuecomment-2507780521
There was the following test failure on amd64 and win64:
```
External error: query result mismatch:
[SQL] select acos(0), acos(0.5), acos(1);
[Diff] (-expected|+actual)
alamb commented on code in PR #13605:
URL: https://github.com/apache/datafusion/pull/13605#discussion_r1863452171
##
datafusion/sqllogictest/test_files/errors.slt:
##
@@ -70,7 +70,7 @@ SELECT COUNT(*) FROM nonexistentschema.aggregate_test_100
statement error Error during planni
Omega359 commented on PR #13404:
URL: https://github.com/apache/datafusion/pull/13404#issuecomment-2507747026
> An alternative approach is that we need to differentiate `string literal`
and `varchar` like Postgres an DuckDB. Only untyped `string literal` is able to
cast to any other types,
Omega359 commented on PR #13404:
URL: https://github.com/apache/datafusion/pull/13404#issuecomment-2507744060
> @Omega359 How about we make this configurable? Enable implicit coercion if
we want the ease of use and the casting cost is acceptable, disable it if we
prefer explicit casting wit
berkaysynnada commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2507725495
I have applied the @blaginin suggestion:
```
berkaysahin@Berkays-MacBook-Pro wasmtest % clang --version
Homebrew clang vers
alamb opened a new pull request, #13605:
URL: https://github.com/apache/datafusion/pull/13605
## Which issue does this PR close?
- Part of https://github.com/apache/datafusion/pull/13546
## Rationale for this change
While working on the upgrade of sqlparser, the structure
timsaucer commented on issue #1142:
URL:
https://github.com/apache/datafusion-ballista/issues/1142#issuecomment-2507706840
But even if that unblocks you I worry it still doesn’t resolve to core issue
of trying to share that session context from one python package to another.
--
This is a
timsaucer commented on issue #1142:
URL:
https://github.com/apache/datafusion-ballista/issues/1142#issuecomment-2507704638
I’ve been meaning to dive into this and also some work happening on
`datafusion-ray` that may encounter similar problems. One thing the
`datafusion-python` package is
alamb merged PR #1556:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1556
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
alamb commented on PR #1556:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1556#issuecomment-2507686514
I double checked with https://github.com/apache/datafusion/pull/13546 and
this works great. Thank you again @goldmedal and @iffyio
--
This is an automated message from t
alamb closed issue #1554: Relation visitor fails to visit the `SHOW COLUMNS`
statement in the latest commit of the main branch
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
ozankabak opened a new pull request, #13604:
URL: https://github.com/apache/datafusion/pull/13604
## Which issue does this PR close?
N/A.
Closes #.
## Rationale for this change
Using succinct test utility functions instead of verbose constructions,
making some dow
alamb commented on issue #1569:
URL:
https://github.com/apache/datafusion-sqlparser-rs/issues/1569#issuecomment-2507640203
Thanks for the report @demetribu -- hopefully this is fixed now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
alamb closed issue #1569: Main branch, linter failure on new Rust version
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
alamb merged PR #1570:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
milenkovicm commented on code in PR #12939:
URL: https://github.com/apache/datafusion/pull/12939#discussion_r1863370988
##
datafusion/sql/src/expr/mod.rs:
##
@@ -1195,4 +1195,25 @@ mod tests {
test_stack_overflow!(2048);
test_stack_overflow!(4096);
test_stack_over
onursatici opened a new pull request, #13603:
URL: https://github.com/apache/datafusion/pull/13603
## Which issue does this PR close?
Closes #.
## Rationale for this change
If a projection is done on a hash partitioned input and if the projection
does not include
ajazam commented on issue #13602:
URL: https://github.com/apache/datafusion/issues/13602#issuecomment-2507538896
I've tried rust 1.82 and 1.83
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
ajazam opened a new issue, #13602:
URL: https://github.com/apache/datafusion/issues/13602
### Describe the bug
I am trying to create a parquet file with hive partitioning, from csv data
and get error
Error: External(NotImplemented("it is not yet supported to write to hive
part
goldmedal opened a new issue, #13601:
URL: https://github.com/apache/datafusion/issues/13601
### Is your feature request related to a problem or challenge?
Given a SQL:
```SQL
select * from unnest([1,2,3]) as t(c1)
```
DataFusion plans the unnest to `Projection/Unnest/Proje
Dandandan commented on issue #13586:
URL: https://github.com/apache/datafusion/issues/13586#issuecomment-2507498291
I didn't profile yet, but one potentially problematic line I found here:
`concat_batches(self.input_schema(), [input_buffer, &record_batch])?`
This concatenates `[
Dandandan commented on issue #13275:
URL: https://github.com/apache/datafusion/issues/13275#issuecomment-2507429313
Oh I saw @LeslieKid already commented on that issue đź‘Ť
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
Dandandan commented on issue #13275:
URL: https://github.com/apache/datafusion/issues/13275#issuecomment-2507428266
Yeah som experiments around that would be nice.
Also see this issue:
https://github.com/apache/datafusion/issues/12131
--
This is an automated message from the Apac
milenkovicm opened a new issue, #1142:
URL: https://github.com/apache/datafusion-ballista/issues/1142
First of all, I'm not expert in rust-python (pyo3) integration, if I've
done/said something stupid,
my apologies.
Current implementation of (py)ballista has limitation when it come
Rachelint commented on issue #13275:
URL: https://github.com/apache/datafusion/issues/13275#issuecomment-2507419912
> Thanks @Rachelint for summarizing, that's interesting. One big difference
between `take` + `eq` in join versus grouped aggregates seems to be that the
`ValueBuilder`s alread
demetribu commented on issue #1569:
URL:
https://github.com/apache/datafusion-sqlparser-rs/issues/1569#issuecomment-2507377938
related #1570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
berkaysynnada commented on PR #13560:
URL: https://github.com/apache/datafusion/pull/13560#issuecomment-2507351353
Hi @haohuaijin, and sorry for the delayed response. I have been very busy
over the past few days. I have reviewed your fix and have some comments about
the problem and the solu
Dandandan commented on issue #13275:
URL: https://github.com/apache/datafusion/issues/13275#issuecomment-2507291597
Thanks @Rachelint for summarizing, that's interesting. One big difference
between `take` + `eq` in join versus grouped aggregates seems to be that the
`ValueBuilder`s already
Groennbeck opened a new pull request, #13600:
URL: https://github.com/apache/datafusion/pull/13600
## Which issue does this PR close?
Closes #.
## Rationale for this change
Want to able to call these functions in apache comet. But cannot create the
expression because
76 matches
Mail list logo
