andygrove opened a new pull request, #1619:
URL: https://github.com/apache/datafusion-comet/pull/1619
## Which issue does this PR close?
N/A
## Rationale for this change
My primary motivation was to be able to run benchmarks with the new scans
with and wi
ding-young commented on issue #15664:
URL: https://github.com/apache/datafusion/issues/15664#issuecomment-2791674000
I think we can refer to the existing microbenchmark for external aggregation
([external_aggr.rs](https://github.com/apache/datafusion/blob/main/benchmarks/src/bin/external_agg
Dandandan commented on PR #15466:
URL: https://github.com/apache/datafusion/pull/15466#issuecomment-2791658055
I wonder if `alamb_test_upgrade_54` has the latest version of 54? Some
performance improvements happened there as well (e.g.
https://github.com/apache/arrow-rs/pull/7195/files shou
iffyio commented on code in PR #1803:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1803#discussion_r2036578066
##
src/tokenizer.rs:
##
@@ -895,7 +895,7 @@ impl<'a> Tokenizer<'a> {
};
let mut location = state.location();
-while let Some
iffyio commented on code in PR #1802:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1802#discussion_r2036520283
##
src/ast/query.rs:
##
@@ -2197,6 +2200,8 @@ pub enum JoinOperator {
match_condition: Expr,
constraint: JoinConstraint,
},
+
iffyio commented on PR #1780:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1780#issuecomment-2791591778
Marking as draft in the meantime as this PR is no longer pending review.
@tomershaniii please feel free to undraft and ping when ready!
--
This is an automated message f
iffyio commented on PR #1793:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1793#issuecomment-2791589818
Marking as draft in the meantime as this is no longer pending review,
@adamchainz please feel free to undraft and ping when ready!
--
This is an automated message from t
iffyio commented on code in PR #1799:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2036518287
##
src/parser/mod.rs:
##
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
-
iffyio commented on code in PR #1799:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2036518287
##
src/parser/mod.rs:
##
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
-
2010YOUY01 commented on PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#issuecomment-2791568836
I tried a simple benchmark:
1. Under `datafusion/datafusion-cli`, compile and run with 100M memory limit
`cargo run --profile release-nonlto -- --mem-pool-type fair -m 100
iffyio merged PR #1801:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1801
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr
alamb opened a new issue, #15664:
URL: https://github.com/apache/datafusion/issues/15664
### Is your feature request related to a problem or challenge?
- Part of https://github.com/apache/datafusion/issues/15271
There are many interesting ideas on how to improve DataFusion whil
brayanjuls commented on issue #15577:
URL: https://github.com/apache/datafusion/issues/15577#issuecomment-2791527017
I was investigating this issue and it seems those tests supposed to be for
PREPARE statements, as the goal when they were implemented was to test infer
types on prepare state
2010YOUY01 commented on PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#issuecomment-2791514957
> And some other thoughts:
>
> 1. This is a pretty complicated program, maybe we should write some unit
tests to make sure it doesn't break for future modifications?
chenkovsky opened a new pull request, #15667:
URL: https://github.com/apache/datafusion/pull/15667
## Which issue does this PR close?
- Closes #15666.
## Rationale for this change
datafusion doesn't support min/max for struct.
## What changes are included in this P
2010YOUY01 commented on PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#issuecomment-2791500223
Thank you all for the review!
@qstommyshu I agree with the implementation-level feedbacks. I will address
them in the refactor.
@alamb Regarding parallel merging: I w
chenkovsky commented on issue #15666:
URL: https://github.com/apache/datafusion/issues/15666#issuecomment-2791497910
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
github-actions[bot] commented on PR #12754:
URL: https://github.com/apache/datafusion/pull/12754#issuecomment-2791365482
Thank you for your contribution. Unfortunately, this pull request is stale
because it has been open 60 days with no activity. Please remove the stale
label or comment or
xudong963 commented on PR #15600:
URL: https://github.com/apache/datafusion/pull/15600#issuecomment-2786981726
> I move out the function and reduce the "stack size" the stack overflow is
gone. `sql_array_literal` is another one example too. `try_from_physical_plan`
has the large function bo
Omega359 commented on PR #15603:
URL: https://github.com/apache/datafusion/pull/15603#issuecomment-2790862097
Thanks for looking into the nullable issue, it's been on my plate for a bit
to look into some more. It's really the last blocker I know of for union by
name to work correctly.
codecov-commenter commented on PR #1619:
URL:
https://github.com/apache/datafusion-comet/pull/1619#issuecomment-2784433520
##
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1619?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca
github-actions[bot] closed pull request #12523: Add hook for sharing join state
in distributed execution
URL: https://github.com/apache/datafusion/pull/12523
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
comphead commented on code in PR #1632:
URL: https://github.com/apache/datafusion-comet/pull/1632#discussion_r2036308811
##
spark/src/test/scala/org/apache/comet/exec/CometNativeReaderSuite.scala:
##
@@ -143,4 +143,96 @@ class CometNativeReaderSuite extends CometTestBase with
A
comphead opened a new pull request, #1632:
URL: https://github.com/apache/datafusion-comet/pull/1632
## Which issue does this PR close?
Related to #1595 .
## Rationale for this change
Adding unit tests for more nested types combinations
## What changes are
lalaorya opened a new issue, #15665:
URL: https://github.com/apache/datafusion/issues/15665
### Describe the bug
When using the LIMIT clause, simple `LIMIT N` syntax (such as `LIMIT 10`)
works normally, but when using the syntax with an offset (such as `LIMIT
10,20`), it fails and re
comphead commented on code in PR #1621:
URL: https://github.com/apache/datafusion-comet/pull/1621#discussion_r2036316165
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -236,16 +171,70 @@ class CometSparkSessionExtensions
CometScanE
comphead commented on code in PR #1632:
URL: https://github.com/apache/datafusion-comet/pull/1632#discussion_r2036308811
##
spark/src/test/scala/org/apache/comet/exec/CometNativeReaderSuite.scala:
##
@@ -143,4 +143,96 @@ class CometNativeReaderSuite extends CometTestBase with
A
comphead commented on issue #1633:
URL:
https://github.com/apache/datafusion-comet/issues/1633#issuecomment-2791245930
To reproduce
```
test("native reader - read STRUCT of MAP fields") {
testSingleLineQuery(
"""
|select named_struct('m0', map('a', 1))
xudong963 commented on code in PR #15539:
URL: https://github.com/apache/datafusion/pull/15539#discussion_r2036278558
##
datafusion/datasource/src/statistics.rs:
##
@@ -410,23 +410,24 @@ pub async fn get_statistics_with_limit(
}
/// Generic function to compute statistics acr
comphead commented on code in PR #1632:
URL: https://github.com/apache/datafusion-comet/pull/1632#discussion_r2036308245
##
spark/src/test/scala/org/apache/comet/exec/CometNativeReaderSuite.scala:
##
@@ -143,4 +143,96 @@ class CometNativeReaderSuite extends CometTestBase with
A
andygrove commented on PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#issuecomment-2791138548
I created a PR in Comet to use DF from this PR -
https://github.com/apache/datafusion-comet/pull/1629
I did not have time to run benchmarks today but hope to tomorrow
--
T
alamb commented on code in PR #67:
URL: https://github.com/apache/datafusion-site/pull/67#discussion_r2036245247
##
content/blog/2025-04-10-fastest-tpch-generator.md:
##
@@ -0,0 +1,613 @@
+---
+layout: post
+title: tpchgen-rs World’s fastest open source TPC-H data generator, wri
parthchandra commented on code in PR #1621:
URL: https://github.com/apache/datafusion-comet/pull/1621#discussion_r2036258826
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -246,6 +183,65 @@ class CometSparkSessionExtensions
}
}
+ pri
andygrove commented on code in PR #1621:
URL: https://github.com/apache/datafusion-comet/pull/1621#discussion_r2036189365
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -246,6 +183,65 @@ class CometSparkSessionExtensions
}
}
+ privat
berkaysynnada commented on PR #15563:
URL: https://github.com/apache/datafusion/pull/15563#issuecomment-2788659010
@geoffreyclaude I'm going to merge this if you're done with this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
alamb commented on code in PR #67:
URL: https://github.com/apache/datafusion-site/pull/67#discussion_r2036251572
##
content/blog/2025-04-10-fastest-tpch-generator.md:
##
@@ -0,0 +1,613 @@
+---
+layout: post
+title: tpchgen-rs World’s fastest open source TPC-H data generator, wri
andygrove merged PR #1601:
URL: https://github.com/apache/datafusion-comet/pull/1601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@
andygrove closed issue #1583: Add documentation for benchmarking Comet in AWS
with S3 data source
URL: https://github.com/apache/datafusion-comet/issues/1583
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
viirya opened a new issue, #22:
URL: https://github.com/apache/datafusion-benchmarks/issues/22
Run `docker build -t datafusion-benchmarks/tpcdsgen .` according to the
README.md under `tpcds`. `tpctools v0.7.0` cannot be built currently.
```
8.751 error: failed to compile `tpctools
andygrove commented on code in PR #1601:
URL: https://github.com/apache/datafusion-comet/pull/1601#discussion_r2036223583
##
docs/source/contributor-guide/benchmarking_aws_ec2.md:
##
@@ -0,0 +1,223 @@
+
+
+# Comet Benchmarking in AWS
+
+This guide is for setting up benchmarks on
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2036221582
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
alamb commented on code in PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#discussion_r2036219770
##
datafusion/physical-plan/src/spill/mod.rs:
##
@@ -24,27 +24,156 @@ use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use std::ptr::Non
alamb commented on PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#issuecomment-2791117066
> > Does anyone know if we have benchmarks for sorting / spilling I could
run to verify the impact of this PR on their behavior?
> > I took a brief look but didn't find any
>
andygrove commented on PR #1614:
URL:
https://github.com/apache/datafusion-comet/pull/1614#issuecomment-2783551229
@comphead @parthchandra could I get a committer review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and us
adriangb commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2791076569
> Maybe it is possible to move the recursion into the optimizer rule but
still keep a `ExecutionPlan` method by making a complex call signature, maybe
something like this:
alamb commented on code in PR #15589:
URL: https://github.com/apache/datafusion/pull/15589#discussion_r2036171313
##
datafusion/sqllogictest/test_files/array.slt:
##
@@ -6140,21 +6140,19 @@ logical_plan
02)--Aggregate: groupBy=[[]], aggr=[[count(Int64(1))]]
03)SubqueryAlia
alamb commented on code in PR #15551:
URL: https://github.com/apache/datafusion/pull/15551#discussion_r2036175257
##
datafusion/expr-common/src/type_coercion/binary.rs:
##
@@ -987,6 +988,25 @@ fn coerce_fields(common_type: DataType, lhs: &FieldRef,
rhs: &FieldRef) -> Field
ashdnazg commented on code in PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#discussion_r2036184085
##
datafusion/common-runtime/src/common.rs:
##
@@ -77,17 +82,32 @@ impl SpawnedTask {
}
}
+impl Future for SpawnedTask {
+type Output = Result;
+
+
shehabgamin commented on code in PR #15588:
URL: https://github.com/apache/datafusion/pull/15588#discussion_r2032217056
##
datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs:
##
@@ -1997,6 +2010,78 @@ fn is_exactly_true(expr: Expr, info: &impl SimplifyInfo)
-> Res
jayzhan211 commented on code in PR #15588:
URL: https://github.com/apache/datafusion/pull/15588#discussion_r2032396391
##
datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs:
##
@@ -1997,6 +2010,78 @@ fn is_exactly_true(expr: Expr, info: &impl SimplifyInfo)
-> Resu
parthchandra commented on code in PR #1624:
URL: https://github.com/apache/datafusion-comet/pull/1624#discussion_r2036139372
##
.github/workflows/spark_sql_test_native_datafusion.yml:
##
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more co
GitHub user qstommyshu added a comment to the discussion: Gathering Ideas for
WASM web playground design
Thanks for your idea @backkem !
> IDK if there is broad enough interest for it but I think querying a remote
> DataFusion instance would be a cool feature. Technically, I think this can be
alamb commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2791020940
Maybe it is possible to move the recursion into the optimizer rule but still
keep a `ExecutionPlan` method by making a complex call signature, maybe
something like this:
```rust
alamb merged PR #15658:
URL: https://github.com/apache/datafusion/pull/15658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2791020563
> It was @alamb that suggested we do it this way, unless I misunderstood his
suggestion.
>
> I think it's possible to do the recursion as an optimizer rule but making
the APIs f
alamb commented on code in PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#discussion_r2036145638
##
datafusion/core/tests/physical_optimizer/filter_pushdown.rs:
##
@@ -0,0 +1,529 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035755891
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,55 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
comphead merged PR #1603:
URL: https://github.com/apache/datafusion-comet/pull/1603
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@d
Dandandan closed pull request #11943: Sketch for aggregation intermediate
results blocked management
URL: https://github.com/apache/datafusion/pull/11943
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
paleolimbot commented on PR #15646:
URL: https://github.com/apache/datafusion/pull/15646#issuecomment-2790938185
I'll take a look this evening!
It's mostly updating to Arrow 55, but
https://github.com/apache/datafusion/pull/15663 (in particular
https://github.com/apache/datafusion/pu
paleolimbot opened a new pull request, #15663:
URL: https://github.com/apache/datafusion/pull/15663
## Which issue does this PR close?
Another experiment in pursuit of
https://github.com/apache/datafusion/issues/12644
## Rationale for this change
It has been suggested th
Dandandan commented on code in PR #11943:
URL: https://github.com/apache/datafusion/pull/11943#discussion_r2036098003
##
datafusion/common/src/config.rs:
##
@@ -338,6 +338,19 @@ config_namespace! {
/// if the source of statistics is accurate.
/// We plan to mak
alamb commented on code in PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#discussion_r2035644581
##
datafusion/common-runtime/src/common.rs:
##
@@ -15,18 +15,23 @@
// specific language governing permissions and limitations
// under the License.
-use std::fut
adriangb commented on code in PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#discussion_r2036084527
##
datafusion/expr/src/filter_pushdown.rs:
##
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agre
dependabot[bot] opened a new pull request, #15627:
URL: https://github.com/apache/datafusion/pull/15627
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.44.1 to 1.44.2.
Release notes
Sourced from https://github.com/tokio-rs/tokio/releases";>tokio's releases.
Tokio v1.4
rluvaton commented on code in PR #14417:
URL: https://github.com/apache/datafusion/pull/14417#discussion_r2035971319
##
datafusion/functions-nested/src/min.rs:
##
@@ -0,0 +1,140 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agr
geoffreyclaude commented on code in PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#discussion_r2036042823
##
datafusion/expr/src/filter_pushdown.rs:
##
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor licens
andygrove opened a new pull request, #15658:
URL: https://github.com/apache/datafusion/pull/15658
## Which issue does this PR close?
N/A
## Rationale for this change
The Java package name needs updating now that DataFusion is a top-level
project.
#
eshed-flarion commented on code in PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#discussion_r2036057402
##
datafusion/common-runtime/src/common.rs:
##
@@ -15,18 +15,23 @@
// specific language governing permissions and limitations
// under the License.
-use
ashdnazg commented on PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#issuecomment-2790857599
Tests should be more reliable now :facepalm:.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
ashdnazg commented on code in PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#discussion_r2036059191
##
datafusion/common-runtime/src/common.rs:
##
@@ -15,18 +15,23 @@
// specific language governing permissions and limitations
// under the License.
-use std::
parthchandra commented on code in PR #1621:
URL: https://github.com/apache/datafusion-comet/pull/1621#discussion_r2036049975
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -246,6 +183,65 @@ class CometSparkSessionExtensions
}
}
+ pri
eshed-flarion commented on code in PR #15653:
URL: https://github.com/apache/datafusion/pull/15653#discussion_r2036057402
##
datafusion/common-runtime/src/common.rs:
##
@@ -15,18 +15,23 @@
// specific language governing permissions and limitations
// under the License.
-use
kumarlokesh commented on code in PR #15644:
URL: https://github.com/apache/datafusion/pull/15644#discussion_r2036056433
##
datafusion/sql/src/parser.rs:
##
@@ -455,9 +457,16 @@ impl<'a> DFParser<'a> {
if let Token::Word(w) =
self.parser.peek_nth_token(1
ozankabak commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2790048758
This is a great contribution and is very close to merge. However, let's make
sure the design is right to avoid API churn. Why do you think
`try_swap_with_projection` is similar? I
Omega359 commented on code in PR #67:
URL: https://github.com/apache/datafusion-site/pull/67#discussion_r2036041501
##
content/blog/2025-04-10-fastest-tpch-generator.md:
##
@@ -0,0 +1,613 @@
+---
+layout: post
+title: tpchgen-rs World’s fastest open source TPC-H data generator,
qstommyshu commented on code in PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#discussion_r2036048151
##
datafusion/physical-plan/src/sorts/sort.rs:
##
@@ -535,56 +457,262 @@ impl ExternalSorter {
// reserved again for the next spill.
self.merg
kumarlokesh commented on code in PR #15644:
URL: https://github.com/apache/datafusion/pull/15644#discussion_r2036043878
##
datafusion/sql/src/parser.rs:
##
@@ -469,17 +478,31 @@ impl<'a> DFParser<'a> {
}
_ => {
/
Omega359 commented on code in PR #67:
URL: https://github.com/apache/datafusion-site/pull/67#discussion_r2036041501
##
content/blog/2025-04-10-fastest-tpch-generator.md:
##
@@ -0,0 +1,613 @@
+---
+layout: post
+title: tpchgen-rs World’s fastest open source TPC-H data generator,
adriangb commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2790824023
For comparison, here is roughly what I had before doing the recursion as
part of the `OptimizerRule`:
https://github.com/pydantic/datafusion/blob/fbf93a2bdd0a5c1532336026dfa71ac7305
qstommyshu commented on code in PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#discussion_r2036031118
##
datafusion/physical-plan/src/sorts/sort.rs:
##
@@ -535,56 +457,262 @@ impl ExternalSorter {
// reserved again for the next spill.
self.merg
qstommyshu commented on code in PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#discussion_r2036027203
##
datafusion/physical-plan/src/sorts/sort.rs:
##
@@ -535,56 +457,262 @@ impl ExternalSorter {
// reserved again for the next spill.
self.merg
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2036022198
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,442 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
qstommyshu commented on code in PR #15610:
URL: https://github.com/apache/datafusion/pull/15610#discussion_r2036013722
##
datafusion/physical-plan/src/sorts/sort.rs:
##
@@ -535,56 +457,262 @@ impl ExternalSorter {
// reserved again for the next spill.
self.merg
adriangb commented on PR #15566:
URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2790778558
Some implementations do recurse I think, for similar reasons to our
recursion here:
https://github.com/pydantic/datafusion/blob/f8a6384bdf21b2eeb7bcfe3f08e52712735bb285/datafusio
rluvaton commented on issue #15662:
URL: https://github.com/apache/datafusion/issues/15662#issuecomment-2790765737
@alamb take:
> > We can start `return_type` deprecation after this PR
>
> I recommend we *do not* deprecate return_type - after this PR I think we
have a nice API. Nam
timsaucer commented on PR #15646:
URL: https://github.com/apache/datafusion/pull/15646#issuecomment-2790756798
I think Aggregate and Window UDFs should come as a separate PR. I did notice
however that for Aggregates the input portion is already viable with this PR.
Since `AccumulatorArgs` a
rluvaton commented on code in PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#discussion_r2035960200
##
datafusion/physical-plan/src/spill/mod.rs:
##
@@ -24,27 +24,156 @@ use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use std::ptr::
andygrove commented on code in PR #1620:
URL: https://github.com/apache/datafusion-comet/pull/1620#discussion_r2035981075
##
.github/actions/setup-spark-builder/action.yaml:
##
@@ -19,13 +19,11 @@ name: Setup Spark Builder
description: 'Setup Apache Spark to run SQL tests'
inp
EmilyMatt commented on PR #15613:
URL: https://github.com/apache/datafusion/pull/15613#issuecomment-2790693147
@alamb Thanks for taking the time to look at this. I've addressed the
relevant points and added some general documentation, I've also modified the
reserved field to an AtomicUsize(
andygrove commented on code in PR #1620:
URL: https://github.com/apache/datafusion-comet/pull/1620#discussion_r2035965475
##
pom.xml:
##
Review Comment:
I tried doing this, but it caused regressions in Spark 4 support, so I'd
prefer to create a separate PR. I filed a track
andygrove opened a new issue, #1631:
URL: https://github.com/apache/datafusion-comet/issues/1631
### What is the problem the feature request solves?
See comment in
https://github.com/apache/datafusion-comet/pull/1620#discussion_r2035772596 for
context
### Describe the potentia
EmilyMatt commented on code in PR #15613:
URL: https://github.com/apache/datafusion/pull/15613#discussion_r2035961703
##
datafusion/execution/src/memory_pool/pool.rs:
##
@@ -261,7 +268,7 @@ fn insufficient_capacity_err(
pub struct TrackConsumersPool {
inner: I,
top: N
EmilyMatt commented on code in PR #15613:
URL: https://github.com/apache/datafusion/pull/15613#discussion_r2035959521
##
datafusion/execution/src/memory_pool/mod.rs:
##
@@ -149,21 +150,65 @@ pub trait MemoryPool: Send + Sync + std::fmt::Debug {
/// For help with allocation acco
rluvaton commented on code in PR #15654:
URL: https://github.com/apache/datafusion/pull/15654#discussion_r2035960200
##
datafusion/physical-plan/src/spill/mod.rs:
##
@@ -24,27 +24,156 @@ use std::fs::File;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use std::ptr::
GitHub user backkem added a comment to the discussion: Gathering Ideas for WASM
web playground design
IDK if there is broad enough interest for it but I think querying a remote
DataFusion instance would be a cool feature. Technically, I think this can be
done by adding [grpc-web](https://gith
romanb opened a new pull request, #1803:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1803
This PR is a follow-up to
https://github.com/apache/datafusion-sqlparser-rs/pull/856. The remaining
problem is that queries with qualified identifiers having numeric prefixes
currently
alamb commented on PR #15655:
URL: https://github.com/apache/datafusion/pull/15655#issuecomment-2790030814
Thanks @phillipleblanc and @xudong963
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
andygrove commented on code in PR #1621:
URL: https://github.com/apache/datafusion-comet/pull/1621#discussion_r2035921195
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -246,6 +183,65 @@ class CometSparkSessionExtensions
}
}
+ privat
alan910127 commented on code in PR #15482:
URL: https://github.com/apache/datafusion/pull/15482#discussion_r2029954233
##
datafusion/sqllogictest/test_files/push_down_filter.slt:
##
@@ -230,19 +230,19 @@ logical_plan TableScan: t projection=[a],
full_filters=[t.a != Int32(100)]
1 - 100 of 257 matches
Mail list logo
